Commit Graph

8496 Commits

Author SHA1 Message Date
Lee Hinman 47856ec4cd Add sandboxing for GString-based method invocation 2014-06-25 12:09:32 +02:00
Shay Banon 342563a864 [LOG] better log message 2014-06-25 11:01:20 +02:00
Alexander Reelsen fd9744968f Internal: Made base64 decode parsing to detect more errors
The base64 did not completely check, if there were other characters
after the equals `=` sign. This PR adds some small additional checks.

Closes #6334
2014-06-24 13:01:11 +02:00
Martijn van Groningen e12025f749 [TEST] Improved logging for replica operation failures 2014-06-23 09:28:41 +02:00
Clinton Gormley 64a4acc49b Docs: Added IDs to the highlighters for linking 2014-06-22 16:46:42 +02:00
Boaz Leskes 3d6d2e700a [Test] testGetFields_complexField should wait for a green cluster
Waiting for ongoing recoveries was not good enough as it can run before the  master finishing processing the started events of primary shards, causing the recovery response to be erroneously empty
2014-06-21 20:15:13 +02:00
Clinton Gormley cf059378d1 Docs: Updated stop token filter docs 2014-06-21 18:42:38 +02:00
Clinton Gormley fac724cc99 Docs: Updated the explanation about memory usage with parent/child 2014-06-21 16:32:29 +02:00
Shay Banon 0e83615496 [Test] Use no failures, shard might not have been initialize yet 2014-06-21 12:43:14 +02:00
Boaz Leskes 08ca51d7b6 [TEST] fix a NPE in verifyThreadNames which may happen if thread finishes during sampling 2014-06-21 10:31:45 +02:00
Shay Banon 0d66d3779e Fix optional default script loading
Groovy is optional as a dependency in the classpath, make sure we properly detect when its not at the right time to disable it
closes #6582
2014-06-21 00:27:15 +02:00
Martijn van Groningen 812972ab0e [TEST] Move the waiting for pending tasks to helper methods and let the percolator and update mapping test use these helper methods. 2014-06-20 23:44:33 +02:00
Martijn van Groningen 11251bca92 [TEST] Verify that all pending tasks are rely executed on *all* nodes. 2014-06-20 23:12:52 +02:00
Martijn van Groningen 73e4a9b3f7 Fixed NPE in recovery api by serializing the recovery type in StartRecoveryRequest.
Closes #6190
2014-06-20 22:09:46 +02:00
javanna f16451a446 Refactored AckedClusterStateUpdateTask & co. to remove code repetitions in subclasses
Made AckedClusterStateUpdateTask an abstract class instead of an interface, which contains the common methods.
Also introduced the AckedRequest interface to mark both AcknowledgedRequest & ClusterStateUpdateRequest so that the different ways of updating the cluster state (with or without a MetaData*Service) can share the same code.
Removed ClusterStateUpdateListener as we can just use its base class ActionListener instead.

Closes #6559
2014-06-20 20:14:40 +02:00
Clinton Gormley e52364a95a Docs: Updated cluster health docs 2014-06-20 18:05:46 +02:00
Lee Hinman 2708e453ac Re-shade MVEL as a dependency 2014-06-20 11:28:50 +02:00
Lee Hinman c70f6d0171 Add Groovy as a scripting language, add sandboxing for Groovy
Sandboxes the groovy scripting language with multiple configurable
whitelists:

`script.groovy.sandbox.receiver_whitelist`: comma-separated list of string
classes for objects that may have methods invoked.
`script.groovy.sandbox.package_whitelist`: comma-separated list of
packages under which new objects may be constructed.
`script.groovy.sandbox.class_whitelist` comma-separated list of classes
that are allowed to be constructed.

As well as a method blacklist:

`script.groovy.sandbox.method_blacklist`: comma-separated list of
methods that are never allowed to be invoked, regardless of target
object.

The sandbox can be entirely disabled by setting:

`script.groovy.sandbox.enabled: false`
2014-06-20 10:20:16 +02:00
javanna 12fd6ce98c REST api: made it possible to copy the REST headers from REST requests to the corresponding TransportRequest(s)
Introduced the use of the FilterClient in all of the REST actions, which delegates all of the operations to the internal Client, but makes sure that the headers are properly copied if needed from REST requests to TransportRequest(s) when it comes to executing them.

Added new abstract handleRequest method to BaseRestHandler with additional Client argument and made private the client instance member (was protected before) to force the use of the client received as argument.

The list of headers to be copied over is by default empty but can be extended via plugins.

Closes #6513
2014-06-19 18:45:21 +02:00
javanna 8f8b2d7979 Client intermediate interface removal follow-up
After #6517 we ended up registering all of the actions (included admin ones) to the NodeClient.
Made sure that only the proper type of Action instances are registered to each client type.
Also fixed some compiler warnings: unused members, imports and non matching generic types.

Closes #6563
2014-06-19 17:55:25 +02:00
Clinton Gormley adf6e794b6 Docs: Rewrote the filtered query docs to be clearer
Closes #1688
2014-06-19 16:34:26 +02:00
Adrien Grand 8ccfca3a2f Fielddata: Remove BytesValues.WithOrdinals.currentOrd and copyShared.
These methods don't exist in Lucene's sorted set doc values.

Relates to #6524
2014-06-19 12:06:40 +02:00
Adrien Grand 9e624942d8 Fielddata: Move `getTermsEnum` from `AtomicFieldData` to `BytesValues.WithOrdinals`.
Similarly to `SortedSetDocValues.termsEnum()`.

Relates to #6524
2014-06-19 12:01:30 +02:00
Adrien Grand 9b02b5061b Fielddata: Merge ordinals APIs into BytesValues.WithOrdinals.
Mid-term we should switch from `BytesValues` to Lucene's doc values APIs, in
particular the `SortedSetDocValues` class. While `BytesValues.WithOrdinals` and
SortedSetDocValues expose the same functionality, `BytesValues.WithOrdinals`
exposes its ordinals via a different `Ordinals.Docs` object while
`SortedSetDocValues` exposes them on the same object as the one that holds the
values. This commit merges ordinals into `BytesValues.WithOrdinals` in order to
make both classes even closer.

Global ordinals were a bit tricky to migrate so I just changed them to use
Lucene's OrdinalMap that will soon (LUCENE-5767, scheduled for 4.9) have the
same optimizations as our global ordinals.

Close #6524
2014-06-19 12:00:51 +02:00
Adrien Grand 703dbff83d Index field names of documents.
The `exists` and `missing` filters need to merge postings lists of all existing
terms, which can be very costly, especially on high-cardinality fields. This
commit indexes the field names of a document under `_field_names` and reuses it
to speed up the `exists` and `missing` filters.

This is only enabled for indices that are created on or after Elasticsearch
1.3.0.

Close #5659
2014-06-19 11:50:06 +02:00
Adrien Grand e2da2114e7 Mappings: Allow _version to use `disk` as a doc values format.
VersionFieldMapper.defaultDocValuesFormat claims that the default is `disk`.
This is not used to choose the DV format in the index but for mappings
serialization in order to know when the _version doc values format is
different from the default format. This made it impossible to use the `disk`
doc values format since mappings would never retain that information at
serialization time.

Close #6523
2014-06-19 11:39:17 +02:00
Boaz Leskes 5b919d4e4f [TEST] Added (trace) logging to testGetFields_complexField 2014-06-19 11:10:13 +02:00
javanna 2024067465 Java API: BulkRequest#add(Iterable) to support UpdateRequests
Closes #6551
2014-06-19 10:43:05 +02:00
Alexander Reelsen 9569166f94 Mapping: Fix possibility of losing meta configuration on field mapping update
The TTL, size, timestamp and index meta properties could be lost on an
update of a single field mapping due to a wrong comparison in the
merge method (which was caused by a wrong initialization, which marked
an update as explicitely disabled instead of unset.

Closes #5053
2014-06-19 08:39:26 +02:00
Fitblip d18fb8bfbd REST API: Allow to configure JSONP/callback support
Added the http.jsonp.enable option to configure disabling of JSONP responses, as those
might pose a security risk, and can be disabled if unused.

This also fixes bugs in NettyHttpChannel
* JSONP responses were never setting application/javascript as the content-type
* The content-type and content-length headers were being overwritten even if they were set before

Closes #6164
2014-06-19 08:34:38 +02:00
Chris 011e20678d [DOCS] Fixed json example in nested-aggregation.asciidoc 2014-06-18 19:38:02 +02:00
Britta Weber 1cbeaf6c45 significant terms: fix json response
Commit fbd7c9aa5d introduced a regression that caused
the min_doc_count to be equal to the number of documents in the
background set. As a result no buckets were built when the
response for significant terms was created.
This only affected the final XContent response.

closes #6535
2014-06-18 18:51:34 +02:00
Zachary Tong ed62f90620 [TEST] Remove indentation on indices.stats test setups
The new indices.stats tests have the setup section indented, which
can cause problems for some yaml parsers.
2014-06-18 11:59:40 -04:00
javanna a499254566 Fixed typo in TransportAction log line 2014-06-18 17:37:45 +02:00
Boaz Leskes 1114835de5 Also send Refresh and Flush actions to relocation targets
Currently we send relocation & flush actions based on all assigned ShardRoutings. During the final stage of relocation, we may miss to refresh/flush a shard if the coordinating node has not yet processed the cluster state update indicating that a relocation is completed *and* the relocation target node has already processed it (i.e., started the shard and has accepted new indexing requests).

Closes #6545
2014-06-18 16:12:42 +02:00
Martijn van Groningen 68046b64c2 [TEST] Only use the clients from the node. 2014-06-18 16:06:09 +02:00
Martijn van Groningen cb97b79396 [TEST] Disable automatic refresh to prevent unintended field data warming of the the _parent field. 2014-06-18 16:06:09 +02:00
Colin Goodheart-Smithe 7423ce0560 Aggregations: Added percentile rank aggregation
Percentile Rank Aggregation is the reverse of the Percetiles aggregation.  It determines the percentile rank (the proportion of values less than a given value) of the provided array of values.

Closes #6386
2014-06-18 12:02:08 +01:00
Britta Weber fe89ea1ecf percolator: fix handling of nested documents
Nested documents were indexed as separate documents, but it was never checked
if the hits represent nested documents or not. Therefore, nested objects could
match not nested queries and nested queries could also match not nested documents.

Examples are in issue #6540 .

closes #6540
closes #6544
2014-06-18 12:24:47 +02:00
Clinton Gormley 69350dc426 Update stemmer-override-tokenfilter.asciidoc 2014-06-18 11:34:20 +02:00
Clinton Gormley 3eb291f334 Docs: tidied configuration.asciidoc 2014-06-17 17:37:07 +02:00
Shay Banon f450c3ea30 update docs to reflect how default write consistency with 1 replica behaves 2014-06-17 14:25:04 +02:00
Martijn van Groningen cb9548f811 Changed the type of field docCounts to IntArray instead of LongArray, because a shard can't hold more than Integer.MAX_VALUE a LongArray just takes unnecessary space.
Closes #6529
2014-06-17 13:35:05 +02:00
Simon Willnauer adb5c19849 [CLIENT] Remove unnecessary intermediate interfaces
Client, ClusterAdminClient and IndicesAdminClient had corresponding
intermediate `internal` interfaces that are unnecessary and cause
a lot of casting. This commit removes the intermediate interfaces
and uses the super interfaces directly.

This commit also adds Releaseable to `Node` and `Client` in order to
be used with utilities like try / with.

Closes #4355
Closes #6517
2014-06-17 12:18:37 +02:00
Simon Willnauer e198c58a6b [TEST] Use test.bwc.version if compatibility version is not present 2014-06-17 12:16:10 +02:00
Britta Weber ef05334fdd don't check .metadata folder for tabs and nocommits 2014-06-17 12:07:13 +02:00
Adrien Grand a06fd46a72 [Benchmark] Fix TermsAggregationSearchBenchmark: The `ordinals` execution mode doesn't exist anymore. 2014-06-17 01:46:02 +02:00
Robert Muir 3892b6ce05 Use ordinals for comparison in GlobalOrdinalsStringTermsAggregator.buildAggregation. Closes #6518 2014-06-16 18:25:28 -04:00
Shay Banon 0427e49b5d [TEST] verify all threads created by node and client have the node name
closes #6516
2014-06-16 21:50:12 +02:00
Martijn van Groningen 612f4618e7 [TEST] wait for ongoing recoveries to finish. Flush fails on shards otherwise. 2014-06-16 17:01:38 +02:00