Commit Graph

5898 Commits

Author SHA1 Message Date
Simon Willnauer f01d3a18ad Simplify FieldData API
Removing #getValue and #hasValue to have a simple and consistent API
for multiple values.
2013-10-31 10:10:35 +01:00
Shay Banon 75eda6e957 upgrade randomized testing to 2.0.12 2013-10-31 10:02:03 +01:00
Alexander Reelsen dfcb3ca2d4 RegexpQueryBuilder now implements MultiTermQueryBuilder
This allows the RegexpQueryBuilder to be used in span queries

Added tests for all span multi term queries.
Also updated the documentation and removed mentioning of numeric range
queries for span queries (they have to be terms).

Closes #3392
2013-10-31 09:12:57 +01:00
Igor Motov d43f3d83d8 The +index pattern without a wildcard in the index list is handled inconsistently
Fixes #3979
2013-10-30 16:36:55 -04:00
Igor Motov 3b4b05e2c9 Fix bug in TransportShardReplicationOperationAction retry mechanism
This issue was causing some index requests against shards in POST_RECOVERY state to hang.
2013-10-30 16:36:55 -04:00
Andrew Raines 5d46e69154 Add ES version to _cat/nodes. 2013-10-30 15:35:41 -05:00
Andrew Raines 256605f065 Add heap used percentage to _cat/nodes.
Closes #4020.
2013-10-30 13:50:29 -05:00
Andrew Raines b2e8ec2924 Add node name & ip to _cat/recovery.
Closes #4016.
2013-10-30 12:38:31 -05:00
Boaz Leskes 8819f91d47 Add a GetFieldMapping API
This new API allows to get the mapping for a specific set of fields rather than get the whole index mapping and traverse it.
The fields to be retrieved can be specified by their full path, index name and field name and will be resolved in this order.
In case multiple field match, the first one will be returned.

Since we are now generating the output (rather then fall back to the stored mapping), you can specify `include_defaults`=true on the request to have default values returned.

Closes #3941
2013-10-30 16:16:36 +01:00
Andrew Raines f16eb7a243 Add _cat/allocation/{nodes}.
Visualizes how shards distribute across the cluster.

% curl localhost:9200/_cat/allocation
shards diskUsed diskAvail diskRatio ip        node
     2   85.6gb   379.4gb     22.6% 127.0.0.1 MacTaggert, Moira
     2   85.6gb   379.4gb     22.6% 127.0.0.1 Jarvis, Edwin

% curl localhost:9200/_cat/allocation/Jar\*
shards diskUsed diskAvail diskRatio ip        node
     2   85.6gb   379.4gb     22.6% 127.0.0.1 Jarvis, Edwin
2013-10-30 10:09:45 -05:00
Shay Banon 8aa2ee7bac indices cluster state service should execute as one of the first listeners
its preferable to execute the indices cluster state service as quickly as possible, as one of the first listeners, so it will apply the cluster state to the local state
potentially, it should even execute before it state is "visible" (through the state call), but that's another change...
2013-10-30 15:53:31 +01:00
Clinton Gormley 8b2efd4849 [DOCS] Added a version flag to percolation 2013-10-30 13:59:03 +01:00
Clinton Gormley 0585890a5f [DOCS] Fixed a typo 2013-10-30 13:57:18 +01:00
Shay Banon 8a62619fb9 throw index already exists failure when it exists as alias on creation
make sure to throw the already exists exception, so when indexing into an alias, and it has not propagated yet through the cluster state, it will end up being ignored if it already exists
2013-10-30 13:47:31 +01:00
Boaz Leskes ed39e79d8f minor improvement to FullRollingRestartTests failure output 2013-10-30 12:08:54 +01:00
Simon Willnauer 454dc53483 Add back hashed BytesValues optimization that got lost in a previous commit.
Some FieldData consumers require hash values per byte. We provide an optimization
that allows to cache the hashes internally if the consumer knows that they are needed
this optimization got lost in a previous commit. This commit adds them back and folds
the dedicated method into AtomicFieldData#getBytesValues(true|false)
2013-10-30 11:48:00 +01:00
Boaz Leskes 542f021d6f Also resend shard started message if shard state is in POST_RECOVERY and master thinks it's initializing.
Closes #4009
2013-10-30 11:42:57 +01:00
Shay Banon 8f21d2831a explicitly enable http when running plugin tests
add an option to force a setting when building a node, overriding if its explicitly set as system property
2013-10-30 02:09:43 +01:00
Luca Cavanna 9cb54c0f2e Added getters for fs.path, fs.mount and fs.dev in node stats api
Closes #4004
2013-10-29 14:49:07 +01:00
Alexander Reelsen 2ec9742147 [DOCS] Extending setup as a service documentation
* Tell people to use ES_JAVA_OPTS for es.node.name or similar parameters
* Showing a simple way to install Oracle JDK on ubuntu/debian

Closes #3999
2013-10-29 13:58:06 +01:00
Alexander Reelsen b6d2be7a51 Improving test stability for CompletionTests
Randomized testing caught an issue showing that the test should
flush after indexing a document, as always the same document is
indexed.
2013-10-29 13:11:34 +01:00
Luca Cavanna 9e6fab3a6d Added support for acknowledgements to update cluster settings api
As a side note, the internal reroute call is now part of the ack mechanism. That means that if the response contains acknowledged flag, the internal reroute that was eventually issued was acknowledged too. Also, even if the request is not acknowledged, the reroute is issued before returning, which means that there is no need to manually call reroute afterwards to make sure the new settings are immediately applied.

Closes #3995
2013-10-29 11:45:59 +01:00
Luca Cavanna d10bd2495d Added support for acknowledgements to cluster reroute api
Closes #3985
2013-10-29 11:45:59 +01:00
Luca Cavanna f23a1e23b1 Added support for node acknowledgements in delete mapping api
Closes #3984
2013-10-29 11:45:59 +01:00
Luca Cavanna 5f1ebf20f7 Added support for acknowledgement in update index settings api
Added support for serialization based on version to AcknowledgedResponse. Useful in api that don't support yet the acknowledged flag in the response.
Moved also ack warmer tests to more specific AckTests class

Close #3983
2013-10-29 11:45:59 +01:00
Luca Cavanna f425d4af2f Added ack call when the cluster state has not changed
The missing call could cause a cluster state update to hang in case there is no change no apply
2013-10-29 11:45:58 +01:00
Luca Cavanna 5639002a8e Fixed test cluster scope to be able to run it multiple times if needed 2013-10-29 11:10:31 +01:00
Igor Motov 01d51ef434 Fix String.format to use Locale.ROOT in RestRecoveryAction 2013-10-28 23:37:11 -04:00
David Pilato 5d90abf701 mget API should support global routing parameter
mget API support `_routing` field but not `routing` parameter.

Reproduction here:

```sh
curl -XDELETE "http://localhost:9200/test/"; echo
curl -XPUT "http://localhost:9200/test/" -d'{
   "settings": {
      "number_of_replicas": 0,
      "number_of_shards": 5
   }
}'; echo

curl -XPUT 'http://localhost:9200/test/order/1-1?routing=key1' -d '{
   "productName":"doc 1"
}'; echo
curl -XPUT 'http://localhost:9200/test/order/1-2?routing=key1' -d '{
   "productName":"doc 2"
}'; echo
curl -XPUT 'http://localhost:9200/test/order/1-3?routing=key1&refresh=true' -d '{
   "productName":"doc 3"
}'; echo

curl -XPOST 'http://localhost:9200/test/order/_mget?pretty' -d '{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "order",
            "_id" : "1-1",
            "_routing" : "key1"
        },
        {
            "_index" : "test",
            "_type" : "order",
            "_id" : "1-2",
            "_routing" : "key1"
        },
        {
            "_index" : "test",
            "_type" : "order",
            "_id" : "1-3",
            "_routing" : "key1"
        }
    ]
}'; echo

curl -XPOST 'http://localhost:9200/test/order/_mget?pretty&routing=key1' -d '{
	"ids": [
		"1-1",
		"1-2",
		"1-3"
	]
}'; echo
```

Closes #3996.
2013-10-28 21:05:55 +01:00
Lee Hinman e3db12bf50 Add the 'recovery' _cat API endpoint
This addes the _cat/recovery/{index} API endpoint, which displays
information about the status of recovering shards. An example of the
output:

index shard node                   target    recovered     %
test2 0     Fwo7c_6MSdWM0uM1Ho4t-g 147304414  19236101 13.1%
test  0     Fwo7c_6MSdWM0uM1Ho4t-g 145891423 119640535 82.0%

Fixes #3969
2013-10-28 13:43:02 -06:00
Alexander Reelsen b1b52b641d Making unit test future proof by using old version, in case versions change 2013-10-28 18:46:46 +01:00
Alexander Reelsen ec0880df45 Add version to prebuilt analyzers
This patch takes the version of the created index into account when a
prebuilt analyzer is created.
So, if an index was created with 0.90.4, then the prebuilt analyzers
will be the same than on the 0.90.4 release.

One reason for this feature is the possibility to change pre built
analyzers like the standard one.

The patch tries to reuse analyzers as mutch as possible. So even if
version X.Y.Z and X.Y.A use the same lucene analyzers, the same instance
is reused in order to prevent overcreation of lucene analyzer instances.

Closes #3790
2013-10-28 18:11:35 +01:00
Britta Weber c9dab6991e rename and document "index.mapping.date.parse_upper_inclusive" setting for date fields
The setting causes the upper bound for a range query/filter to be rounded up,
therefore the name `round_ceil` seems to make more sense.

Also this commit removes the redundant fourth parameter to DateMathParser.parse(..)
which was never used.
was:    parse(String text, long now, boolean roundUp, boolean upperInclusive)
is now: parse(String text, long now, boolean roundCeil)

closes #3914
2013-10-28 15:48:31 +01:00
Adrien Grand 2e8bbe9e30 Make ContextIndexSearcher delegate more method calls to Engine.searcher().
When running tests, Engine.searcher() is going to be an AssertingIndexSearcher
so we definitely don't want to discard it. This commit fixes it as well as the
bugs it found.

Closes #3987
2013-10-28 13:12:09 +01:00
Simon Willnauer ce891e93b6 Assert on failures in QueryRescorerTests 2013-10-28 11:09:48 +01:00
Boaz Leskes 6b52391dbd Added a ClusterHealthResponse.toString implementation for better error reporting in FullRollingRestartTests timeout failures. 2013-10-28 09:54:54 +01:00
Boaz Leskes 68de46ff05 Use XContentParser.isBooleanValue to detect all boolean values for the `_source` parameter for both _mget and _search
Closes #3981
2013-10-26 22:19:05 +02:00
Boaz Leskes 0f8149dca4 Added Booleans.isBoolean to allow detecting whether a char sequence contains a boolean value. Added XContentParser.isBooleanValue to allow testing if current value is of boolean nature.
Boolean values:
boolean type values
numeric (0 == false)
one of the following strings: "true","false","on","off","yes","no","0","1"
2013-10-26 22:15:35 +02:00
Simon Willnauer 2255ecc980 Don't use TransportClient on FullClusterRestartTests
There seems to be an issue with this test since it shuts down random
nodes and TransportClients seem to be confused due to that. For
now we disable them to figure out if this is the cause of the sporadic
timeouts.
2013-10-25 22:04:44 +02:00
Adrien Grand b0b3748cae Re-apply pull-request #3956 now that the CLA has been signed. 2013-10-25 19:05:20 +02:00
Alexander Reelsen f9154de04f CompletionFieldMapper: Return error if weight is no integer
In order to make sure that people do not get confused, if they
index a float as weight, it makes more sense to reject it instead of
silently parsing it to an integer and using it.

The CompletionFieldMapper now checks for the type of the number which
is being read and throws and exception if the number is something else
than int or long.

Closes #3977
2013-10-25 16:38:50 +02:00
Simon Willnauer e79f714b4b Unify REST responses for Acknowledged[Request|Response]
All Rest handlers that use AcknowledgedResponse look identical. Since
we gonna have more of them in the future we can abstract the logic away.
2013-10-25 14:31:28 +02:00
Helton Alponti 0e9c049da1 Fixed NPE in multi_match query when using lenient and field weight
Closes #3797
2013-10-25 00:49:17 +02:00
Ben McCann cc4bc7d57d Fix nonsensical sentence in standard analyzer documentation so that it is more understandable 2013-10-25 00:18:32 +02:00
Luca Cavanna 48ac9747a8 Added third highlighter type based on lucene postings highlighter
Requires field index_options set to "offsets" in order to store positions and offsets in the postings list.
Considerably faster than the plain highlighter since it doesn't require to reanalyze the text to be highlighted: the larger the documents the better the performance gain should be.
Requires less disk space than term_vectors, needed for the fast_vector_highlighter.
Breaks the text into sentences and highlights them. Uses a BreakIterator to find sentences in the text. Plays really well with natural text, not quite the same if the text contains html markup for instance.
Treats the document as the whole corpus, and scores individual sentences as if they were documents in this corpus, using the BM25 algorithm.

Uses forked version of lucene postings highlighter to support:
- per value discrete highlighting for fields that have multiple values, needed when number_of_fragments=0 since we want to return a snippet per value
- manually passing in query terms to avoid calling extract terms multiple times, since we use a different highlighter instance per doc/field, but the query is always the same

The lucene postings highlighter api is  quite different compared to the existing highlighters api, the main difference being that it allows to highlight multiple fields in multiple docs with a single call, ensuring sequential IO.
The way it is introduced in elasticsearch in this first round is a compromise trying not to change the current highlight api, which works per document, per field. The main disadvantage is that we lose the sequential IO, but we can always refactor the highlight api to work with multiple documents.

Supports pre_tag, post_tag, number_of_fragments (0 highlights the whole field), require_field_match, no_match_size, order by score and html encoding.

Closes #3704
2013-10-24 23:38:00 +02:00
Simon Willnauer 08b4ca66c3 Use at least one query to prevent division by zero in PercolatorFacetsTests 2013-10-24 21:45:12 +02:00
Britta Weber ebd328e340 remove trove import from mvel script
all subclasses of trove maps have been removed in 088e05b368
2013-10-24 19:44:45 +02:00
Simon Willnauer 3a34aa735e Upgrade to Lucene 4.5.1 2013-10-24 18:37:44 +02:00
Luca Cavanna e981e411d7 [DOCS] rephrased docs for highlight no_match_size parameter
(removed 0.90.6 coming tag as it's needed only in 0.90 branch)
2013-10-24 14:38:32 +02:00
Luca Cavanna 8e6c4ce8e8 Minor changes to no_match_size highlight parameter and highlight tests (#1171)
- Randomly store the field to highlight
- Updated test to use common assertions
- Restored previously commented out testCommonTermsQuery
2013-10-24 14:38:32 +02:00