Commit Graph

9669 Commits

Author SHA1 Message Date
Boaz Leskes a72c167be2 [Test] improved recovery slow down in rerouteRecoveryTest
only change recovery throttling to slow down recoveries. The recovery file chunk size updates are not picked up by ongoing recoveries. That cause the recovery to take too long even after the default settings are restored.

Also - change document creation to reuse field names in order to speed up the test.
2014-06-29 14:37:12 +02:00
Boaz Leskes bbc82e2821 [Test] add awaitFix to rerouteRecoveryTest 2014-06-29 09:55:03 +02:00
Boaz Leskes ca194594b3 Recovery API should also report ongoing relocation recoveries
We currently only report relocation related recoveries after they are done.

Closes #6585
2014-06-28 21:27:15 +02:00
Boaz Leskes 155620ed8e [Test] testRelocationWhileRefreshing should wait for the first shard to be started 2014-06-28 10:41:06 +02:00
Simon Willnauer 9ce66cb167 [TEST] Testcase for #6639 2014-06-28 09:12:25 +02:00
Simon Willnauer 309c7ceeff Added minimal setup guide for BW Compat tests 2014-06-27 15:39:53 +02:00
Robert Muir a3d5381392 Disable explicit GC by default
We don't rely upon GC to cleanup mappedbytebuffers, we unmap them
explicitly on close in lucene. But the JDK has crazy loops with
explicit GCs in exceptional cases to try to force unmapping.

In general we don't want any of our code or library code calling
this method: so its banned in forbidden-apis as well.
2014-06-27 14:09:44 +02:00
Simon Willnauer b2685f132a [TEST] Change es.node.mode default for tests to `local`
In order to speed up test execution we should run in local mode by
default. CI builds will still use network builds all the time.

Closes #6624
2014-06-27 11:57:34 +02:00
Simon Willnauer f0cfdc444f [STORE] Wrap RateLimiter rather than copy RateLimitedIndexOutput
We clone RateLimitedIndexOutput from lucene just to collect pausing
statistics we can do this in a more straight forward way in a delegating
RateLimiter.

Closes #6625
2014-06-27 11:35:13 +02:00
Shay Banon 79af3228ad Thread pool rejection status code should be 429
Thread rejection should return too many requests status code, and not 503, which is used to also show that the cluster is not available
 relates to #6627, but only for rejections for now
closes #6629
2014-06-27 11:15:16 +02:00
Shay Banon 4129bb6a4f Make sure we don't reuse arrays when sending and error back
We want to make sure recycling will not fail for any reason while trying to send a response back that is caused by a failure, for example, if we have circuit breaker on it (at one point), sending an error back will not be affected by it.
closes #6631
2014-06-27 11:12:35 +02:00
Shay Banon e559295228 [TEST] when the test fail, have the exception message as the reason
the test failed but couldn't repro (yet), at the very least, make sure we have the exception message as the reason, can help to track down the failure itself when it happens again
2014-06-27 09:16:51 +02:00
Simon Willnauer f7da6da73a [TEST] suppress sysout checks since CI runs with debug enabled 2014-06-26 19:10:20 +02:00
Robert Muir b55ad98d73 Upgrade to Lucene 4.9 (closes #6623) 2014-06-26 08:18:59 -04:00
Lee Hinman b43b56a6a8 Add a transformer to translate constant BigDecimal to double 2014-06-26 10:52:28 +02:00
Lee Hinman 50bb274efa Remove MVEL as a built-in scripting language 2014-06-26 10:33:28 +02:00
mahdeto e78f1edca3 DOC:Added field data circuit breaker settings 2014-06-26 10:29:41 +02:00
Boaz Leskes 2c2783875e Be more diligent about ThreadPools having names
Add a name parameter to what was the empty ThreadPool constructor. Assert if the the ThreadPool's setting doesn't contain a name.
2014-06-26 10:01:22 +02:00
Clinton Gormley 30c80319c0 Match query with operator and, cutoff_frequency and stacked tokens
If the match query with cutoff_frequency encounters stacked tokens,
like synonyms in the same position, it returns a boolean query instead
of a common terms query.  However, if the original operator was set
to "and", it was ignoring that and resetting the operator to "or".

In fact, if operator is "and" then there is little benefit in using
a common terms query as a must query is already
executed efficiently.
2014-06-25 17:53:43 +02:00
Andrew Raines 534b07a3fb [TEST] Add assertion failure messages 2014-06-25 16:22:20 +02:00
Lee Hinman 5c6d28240f Switch to Groovy as the default scripting language
This is a breaking change to move from MVEL -> Groovy
2014-06-25 12:15:12 +02:00
Lee Hinman 47856ec4cd Add sandboxing for GString-based method invocation 2014-06-25 12:09:32 +02:00
Shay Banon 342563a864 [LOG] better log message 2014-06-25 11:01:20 +02:00
Alexander Reelsen fd9744968f Internal: Made base64 decode parsing to detect more errors
The base64 did not completely check, if there were other characters
after the equals `=` sign. This PR adds some small additional checks.

Closes #6334
2014-06-24 13:01:11 +02:00
Martijn van Groningen e12025f749 [TEST] Improved logging for replica operation failures 2014-06-23 09:28:41 +02:00
Clinton Gormley 64a4acc49b Docs: Added IDs to the highlighters for linking 2014-06-22 16:46:42 +02:00
Boaz Leskes 3d6d2e700a [Test] testGetFields_complexField should wait for a green cluster
Waiting for ongoing recoveries was not good enough as it can run before the  master finishing processing the started events of primary shards, causing the recovery response to be erroneously empty
2014-06-21 20:15:13 +02:00
Clinton Gormley cf059378d1 Docs: Updated stop token filter docs 2014-06-21 18:42:38 +02:00
Clinton Gormley fac724cc99 Docs: Updated the explanation about memory usage with parent/child 2014-06-21 16:32:29 +02:00
Shay Banon 0e83615496 [Test] Use no failures, shard might not have been initialize yet 2014-06-21 12:43:14 +02:00
Boaz Leskes 08ca51d7b6 [TEST] fix a NPE in verifyThreadNames which may happen if thread finishes during sampling 2014-06-21 10:31:45 +02:00
Shay Banon 0d66d3779e Fix optional default script loading
Groovy is optional as a dependency in the classpath, make sure we properly detect when its not at the right time to disable it
closes #6582
2014-06-21 00:27:15 +02:00
Martijn van Groningen 812972ab0e [TEST] Move the waiting for pending tasks to helper methods and let the percolator and update mapping test use these helper methods. 2014-06-20 23:44:33 +02:00
Martijn van Groningen 11251bca92 [TEST] Verify that all pending tasks are rely executed on *all* nodes. 2014-06-20 23:12:52 +02:00
Martijn van Groningen 73e4a9b3f7 Fixed NPE in recovery api by serializing the recovery type in StartRecoveryRequest.
Closes #6190
2014-06-20 22:09:46 +02:00
javanna f16451a446 Refactored AckedClusterStateUpdateTask & co. to remove code repetitions in subclasses
Made AckedClusterStateUpdateTask an abstract class instead of an interface, which contains the common methods.
Also introduced the AckedRequest interface to mark both AcknowledgedRequest & ClusterStateUpdateRequest so that the different ways of updating the cluster state (with or without a MetaData*Service) can share the same code.
Removed ClusterStateUpdateListener as we can just use its base class ActionListener instead.

Closes #6559
2014-06-20 20:14:40 +02:00
Clinton Gormley e52364a95a Docs: Updated cluster health docs 2014-06-20 18:05:46 +02:00
Lee Hinman 2708e453ac Re-shade MVEL as a dependency 2014-06-20 11:28:50 +02:00
Lee Hinman c70f6d0171 Add Groovy as a scripting language, add sandboxing for Groovy
Sandboxes the groovy scripting language with multiple configurable
whitelists:

`script.groovy.sandbox.receiver_whitelist`: comma-separated list of string
classes for objects that may have methods invoked.
`script.groovy.sandbox.package_whitelist`: comma-separated list of
packages under which new objects may be constructed.
`script.groovy.sandbox.class_whitelist` comma-separated list of classes
that are allowed to be constructed.

As well as a method blacklist:

`script.groovy.sandbox.method_blacklist`: comma-separated list of
methods that are never allowed to be invoked, regardless of target
object.

The sandbox can be entirely disabled by setting:

`script.groovy.sandbox.enabled: false`
2014-06-20 10:20:16 +02:00
javanna 12fd6ce98c REST api: made it possible to copy the REST headers from REST requests to the corresponding TransportRequest(s)
Introduced the use of the FilterClient in all of the REST actions, which delegates all of the operations to the internal Client, but makes sure that the headers are properly copied if needed from REST requests to TransportRequest(s) when it comes to executing them.

Added new abstract handleRequest method to BaseRestHandler with additional Client argument and made private the client instance member (was protected before) to force the use of the client received as argument.

The list of headers to be copied over is by default empty but can be extended via plugins.

Closes #6513
2014-06-19 18:45:21 +02:00
javanna 8f8b2d7979 Client intermediate interface removal follow-up
After #6517 we ended up registering all of the actions (included admin ones) to the NodeClient.
Made sure that only the proper type of Action instances are registered to each client type.
Also fixed some compiler warnings: unused members, imports and non matching generic types.

Closes #6563
2014-06-19 17:55:25 +02:00
Clinton Gormley adf6e794b6 Docs: Rewrote the filtered query docs to be clearer
Closes #1688
2014-06-19 16:34:26 +02:00
Adrien Grand 8ccfca3a2f Fielddata: Remove BytesValues.WithOrdinals.currentOrd and copyShared.
These methods don't exist in Lucene's sorted set doc values.

Relates to #6524
2014-06-19 12:06:40 +02:00
Adrien Grand 9e624942d8 Fielddata: Move `getTermsEnum` from `AtomicFieldData` to `BytesValues.WithOrdinals`.
Similarly to `SortedSetDocValues.termsEnum()`.

Relates to #6524
2014-06-19 12:01:30 +02:00
Adrien Grand 9b02b5061b Fielddata: Merge ordinals APIs into BytesValues.WithOrdinals.
Mid-term we should switch from `BytesValues` to Lucene's doc values APIs, in
particular the `SortedSetDocValues` class. While `BytesValues.WithOrdinals` and
SortedSetDocValues expose the same functionality, `BytesValues.WithOrdinals`
exposes its ordinals via a different `Ordinals.Docs` object while
`SortedSetDocValues` exposes them on the same object as the one that holds the
values. This commit merges ordinals into `BytesValues.WithOrdinals` in order to
make both classes even closer.

Global ordinals were a bit tricky to migrate so I just changed them to use
Lucene's OrdinalMap that will soon (LUCENE-5767, scheduled for 4.9) have the
same optimizations as our global ordinals.

Close #6524
2014-06-19 12:00:51 +02:00
Adrien Grand 703dbff83d Index field names of documents.
The `exists` and `missing` filters need to merge postings lists of all existing
terms, which can be very costly, especially on high-cardinality fields. This
commit indexes the field names of a document under `_field_names` and reuses it
to speed up the `exists` and `missing` filters.

This is only enabled for indices that are created on or after Elasticsearch
1.3.0.

Close #5659
2014-06-19 11:50:06 +02:00
Adrien Grand e2da2114e7 Mappings: Allow _version to use `disk` as a doc values format.
VersionFieldMapper.defaultDocValuesFormat claims that the default is `disk`.
This is not used to choose the DV format in the index but for mappings
serialization in order to know when the _version doc values format is
different from the default format. This made it impossible to use the `disk`
doc values format since mappings would never retain that information at
serialization time.

Close #6523
2014-06-19 11:39:17 +02:00
Boaz Leskes 5b919d4e4f [TEST] Added (trace) logging to testGetFields_complexField 2014-06-19 11:10:13 +02:00
javanna 2024067465 Java API: BulkRequest#add(Iterable) to support UpdateRequests
Closes #6551
2014-06-19 10:43:05 +02:00
Alexander Reelsen 9569166f94 Mapping: Fix possibility of losing meta configuration on field mapping update
The TTL, size, timestamp and index meta properties could be lost on an
update of a single field mapping due to a wrong comparison in the
merge method (which was caused by a wrong initialization, which marked
an update as explicitely disabled instead of unset.

Closes #5053
2014-06-19 08:39:26 +02:00