Commit Graph

5932 Commits

Author SHA1 Message Date
Boaz Leskes 542f021d6f Also resend shard started message if shard state is in POST_RECOVERY and master thinks it's initializing.
Closes #4009
2013-10-30 11:42:57 +01:00
Shay Banon 8f21d2831a explicitly enable http when running plugin tests
add an option to force a setting when building a node, overriding if its explicitly set as system property
2013-10-30 02:09:43 +01:00
Luca Cavanna 9cb54c0f2e Added getters for fs.path, fs.mount and fs.dev in node stats api
Closes #4004
2013-10-29 14:49:07 +01:00
Alexander Reelsen 2ec9742147 [DOCS] Extending setup as a service documentation
* Tell people to use ES_JAVA_OPTS for es.node.name or similar parameters
* Showing a simple way to install Oracle JDK on ubuntu/debian

Closes #3999
2013-10-29 13:58:06 +01:00
Alexander Reelsen b6d2be7a51 Improving test stability for CompletionTests
Randomized testing caught an issue showing that the test should
flush after indexing a document, as always the same document is
indexed.
2013-10-29 13:11:34 +01:00
Luca Cavanna 9e6fab3a6d Added support for acknowledgements to update cluster settings api
As a side note, the internal reroute call is now part of the ack mechanism. That means that if the response contains acknowledged flag, the internal reroute that was eventually issued was acknowledged too. Also, even if the request is not acknowledged, the reroute is issued before returning, which means that there is no need to manually call reroute afterwards to make sure the new settings are immediately applied.

Closes #3995
2013-10-29 11:45:59 +01:00
Luca Cavanna d10bd2495d Added support for acknowledgements to cluster reroute api
Closes #3985
2013-10-29 11:45:59 +01:00
Luca Cavanna f23a1e23b1 Added support for node acknowledgements in delete mapping api
Closes #3984
2013-10-29 11:45:59 +01:00
Luca Cavanna 5f1ebf20f7 Added support for acknowledgement in update index settings api
Added support for serialization based on version to AcknowledgedResponse. Useful in api that don't support yet the acknowledged flag in the response.
Moved also ack warmer tests to more specific AckTests class

Close #3983
2013-10-29 11:45:59 +01:00
Luca Cavanna f425d4af2f Added ack call when the cluster state has not changed
The missing call could cause a cluster state update to hang in case there is no change no apply
2013-10-29 11:45:58 +01:00
Luca Cavanna 5639002a8e Fixed test cluster scope to be able to run it multiple times if needed 2013-10-29 11:10:31 +01:00
Igor Motov 01d51ef434 Fix String.format to use Locale.ROOT in RestRecoveryAction 2013-10-28 23:37:11 -04:00
David Pilato 5d90abf701 mget API should support global routing parameter
mget API support `_routing` field but not `routing` parameter.

Reproduction here:

```sh
curl -XDELETE "http://localhost:9200/test/"; echo
curl -XPUT "http://localhost:9200/test/" -d'{
   "settings": {
      "number_of_replicas": 0,
      "number_of_shards": 5
   }
}'; echo

curl -XPUT 'http://localhost:9200/test/order/1-1?routing=key1' -d '{
   "productName":"doc 1"
}'; echo
curl -XPUT 'http://localhost:9200/test/order/1-2?routing=key1' -d '{
   "productName":"doc 2"
}'; echo
curl -XPUT 'http://localhost:9200/test/order/1-3?routing=key1&refresh=true' -d '{
   "productName":"doc 3"
}'; echo

curl -XPOST 'http://localhost:9200/test/order/_mget?pretty' -d '{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "order",
            "_id" : "1-1",
            "_routing" : "key1"
        },
        {
            "_index" : "test",
            "_type" : "order",
            "_id" : "1-2",
            "_routing" : "key1"
        },
        {
            "_index" : "test",
            "_type" : "order",
            "_id" : "1-3",
            "_routing" : "key1"
        }
    ]
}'; echo

curl -XPOST 'http://localhost:9200/test/order/_mget?pretty&routing=key1' -d '{
	"ids": [
		"1-1",
		"1-2",
		"1-3"
	]
}'; echo
```

Closes #3996.
2013-10-28 21:05:55 +01:00
Lee Hinman e3db12bf50 Add the 'recovery' _cat API endpoint
This addes the _cat/recovery/{index} API endpoint, which displays
information about the status of recovering shards. An example of the
output:

index shard node                   target    recovered     %
test2 0     Fwo7c_6MSdWM0uM1Ho4t-g 147304414  19236101 13.1%
test  0     Fwo7c_6MSdWM0uM1Ho4t-g 145891423 119640535 82.0%

Fixes #3969
2013-10-28 13:43:02 -06:00
Alexander Reelsen b1b52b641d Making unit test future proof by using old version, in case versions change 2013-10-28 18:46:46 +01:00
Alexander Reelsen ec0880df45 Add version to prebuilt analyzers
This patch takes the version of the created index into account when a
prebuilt analyzer is created.
So, if an index was created with 0.90.4, then the prebuilt analyzers
will be the same than on the 0.90.4 release.

One reason for this feature is the possibility to change pre built
analyzers like the standard one.

The patch tries to reuse analyzers as mutch as possible. So even if
version X.Y.Z and X.Y.A use the same lucene analyzers, the same instance
is reused in order to prevent overcreation of lucene analyzer instances.

Closes #3790
2013-10-28 18:11:35 +01:00
Britta Weber c9dab6991e rename and document "index.mapping.date.parse_upper_inclusive" setting for date fields
The setting causes the upper bound for a range query/filter to be rounded up,
therefore the name `round_ceil` seems to make more sense.

Also this commit removes the redundant fourth parameter to DateMathParser.parse(..)
which was never used.
was:    parse(String text, long now, boolean roundUp, boolean upperInclusive)
is now: parse(String text, long now, boolean roundCeil)

closes #3914
2013-10-28 15:48:31 +01:00
Adrien Grand 2e8bbe9e30 Make ContextIndexSearcher delegate more method calls to Engine.searcher().
When running tests, Engine.searcher() is going to be an AssertingIndexSearcher
so we definitely don't want to discard it. This commit fixes it as well as the
bugs it found.

Closes #3987
2013-10-28 13:12:09 +01:00
Simon Willnauer ce891e93b6 Assert on failures in QueryRescorerTests 2013-10-28 11:09:48 +01:00
Boaz Leskes 6b52391dbd Added a ClusterHealthResponse.toString implementation for better error reporting in FullRollingRestartTests timeout failures. 2013-10-28 09:54:54 +01:00
Boaz Leskes 68de46ff05 Use XContentParser.isBooleanValue to detect all boolean values for the `_source` parameter for both _mget and _search
Closes #3981
2013-10-26 22:19:05 +02:00
Boaz Leskes 0f8149dca4 Added Booleans.isBoolean to allow detecting whether a char sequence contains a boolean value. Added XContentParser.isBooleanValue to allow testing if current value is of boolean nature.
Boolean values:
boolean type values
numeric (0 == false)
one of the following strings: "true","false","on","off","yes","no","0","1"
2013-10-26 22:15:35 +02:00
Simon Willnauer 2255ecc980 Don't use TransportClient on FullClusterRestartTests
There seems to be an issue with this test since it shuts down random
nodes and TransportClients seem to be confused due to that. For
now we disable them to figure out if this is the cause of the sporadic
timeouts.
2013-10-25 22:04:44 +02:00
Adrien Grand b0b3748cae Re-apply pull-request #3956 now that the CLA has been signed. 2013-10-25 19:05:20 +02:00
Alexander Reelsen f9154de04f CompletionFieldMapper: Return error if weight is no integer
In order to make sure that people do not get confused, if they
index a float as weight, it makes more sense to reject it instead of
silently parsing it to an integer and using it.

The CompletionFieldMapper now checks for the type of the number which
is being read and throws and exception if the number is something else
than int or long.

Closes #3977
2013-10-25 16:38:50 +02:00
Simon Willnauer e79f714b4b Unify REST responses for Acknowledged[Request|Response]
All Rest handlers that use AcknowledgedResponse look identical. Since
we gonna have more of them in the future we can abstract the logic away.
2013-10-25 14:31:28 +02:00
Helton Alponti 0e9c049da1 Fixed NPE in multi_match query when using lenient and field weight
Closes #3797
2013-10-25 00:49:17 +02:00
Ben McCann cc4bc7d57d Fix nonsensical sentence in standard analyzer documentation so that it is more understandable 2013-10-25 00:18:32 +02:00
Luca Cavanna 48ac9747a8 Added third highlighter type based on lucene postings highlighter
Requires field index_options set to "offsets" in order to store positions and offsets in the postings list.
Considerably faster than the plain highlighter since it doesn't require to reanalyze the text to be highlighted: the larger the documents the better the performance gain should be.
Requires less disk space than term_vectors, needed for the fast_vector_highlighter.
Breaks the text into sentences and highlights them. Uses a BreakIterator to find sentences in the text. Plays really well with natural text, not quite the same if the text contains html markup for instance.
Treats the document as the whole corpus, and scores individual sentences as if they were documents in this corpus, using the BM25 algorithm.

Uses forked version of lucene postings highlighter to support:
- per value discrete highlighting for fields that have multiple values, needed when number_of_fragments=0 since we want to return a snippet per value
- manually passing in query terms to avoid calling extract terms multiple times, since we use a different highlighter instance per doc/field, but the query is always the same

The lucene postings highlighter api is  quite different compared to the existing highlighters api, the main difference being that it allows to highlight multiple fields in multiple docs with a single call, ensuring sequential IO.
The way it is introduced in elasticsearch in this first round is a compromise trying not to change the current highlight api, which works per document, per field. The main disadvantage is that we lose the sequential IO, but we can always refactor the highlight api to work with multiple documents.

Supports pre_tag, post_tag, number_of_fragments (0 highlights the whole field), require_field_match, no_match_size, order by score and html encoding.

Closes #3704
2013-10-24 23:38:00 +02:00
Simon Willnauer 08b4ca66c3 Use at least one query to prevent division by zero in PercolatorFacetsTests 2013-10-24 21:45:12 +02:00
Britta Weber ebd328e340 remove trove import from mvel script
all subclasses of trove maps have been removed in 088e05b368
2013-10-24 19:44:45 +02:00
Simon Willnauer 3a34aa735e Upgrade to Lucene 4.5.1 2013-10-24 18:37:44 +02:00
Luca Cavanna e981e411d7 [DOCS] rephrased docs for highlight no_match_size parameter
(removed 0.90.6 coming tag as it's needed only in 0.90 branch)
2013-10-24 14:38:32 +02:00
Luca Cavanna 8e6c4ce8e8 Minor changes to no_match_size highlight parameter and highlight tests (#1171)
- Randomly store the field to highlight
- Updated test to use common assertions
- Restored previously commented out testCommonTermsQuery
2013-10-24 14:38:32 +02:00
Nik Everett 14a709f563 Highlighting can return excerpt with no highlights
You can configure the highlighting api to return an excerpt of a field
even if there wasn't a match on the field.

The FVH makes excerpts from the beginning of the string to the first
boundary character after the requested length or the boundary_max_scan,
whichever comes first.  The Plain highlighter makes excerpts from the
beginning of the string to the end of the last token before the requested
length.

Closes #1171
2013-10-24 14:38:32 +02:00
Costin Leau 919720ab4f add detection of JRE server (JRE on Windows Server)
fixes #3928

(cherry picked from commit a176ffda6fd97b6efbce6cc4f02a824bf5a10a17)
2013-10-24 15:28:17 +03:00
Costin Leau 64e4883e2e add stop timeout and start mode for windows service.bat
fixes #3938
fixes #3962

(cherry picked from commit f1d6ccc5845ca8f93177c0de3da59a2f37f67818)
2013-10-24 15:27:32 +03:00
Simon Willnauer fb9cd5a562 Use abstract classes as super typs for Acknowledge operations
Currently we have a marker interface for Acknowledged[Request|Response],
this makes not much sense since we duplicate the code in each subclass
or class that implements the interface. We can simply use abstract
classes and have it implemented only once.
2013-10-24 14:01:43 +02:00
Simon Willnauer 7867de4f5b Refactor FieldData iterations
This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter
into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require
a auxillary class (Iter) but instead driven by native for loops. All
[Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId`
and `nextValue` methods to iterate over all values in a document.
This has several advantage:

 * The amout of specialized classes is reduced
 * Iteration is clearly stateful ie. Iters can't be confused to be local.
 * All iterations are size bounded which prevents runtime checks and
   allows JIT optimizations / loop un-rolling and most iterations are
   branch free.
 * Due to the bounded iteration the need for a `hasNext` method call
   is removed.
 * Value iterations feels more native.

This commit also adds consistent documentation and unifies the calcualtion
if SortMode is involved.

This commit also changes the runtime behavior of BytesValues#getValue() such that it
will never return `null` anymore. If a document has no value in a field
this method still returns a `BytesRef` with a `length` of 0. To identify
documents with no values #hasValue() or #setDocument(int) should be used.
The latter should be preferred if the value will be consumed in the case
the document has a value.
2013-10-24 10:33:56 +02:00
Adrien Grand 7bd1a55f6e Revert "Fix for has_child can cause an infinite loop (100% CPU) when used in bool query."
Temporarily revert the commit while waiting for the CLA to be signed.
2013-10-24 09:43:58 +02:00
Josh Canfield adadc72da3 Fix for has_child can cause an infinite loop (100% CPU) when used in bool query.
Closes #3955
2013-10-24 09:24:14 +02:00
Shay Banon 35b573ff24 Transport: Have a separate channel for recovery
Have a separate channel for recovery, so it won't overflow the "low" channel which is also used for bulk indexing.

Also, rename the channel names to be more descriptive. Change low to bulk (for bulk based operations, currently just bulk indexing), med to reg (for "regular" operations), and high to state (for state based communication). The new channel for recovery will be named recovery, and the ping channel will remain the same.

closes #3954
2013-10-23 15:55:27 -07:00
Adrien Grand d18192b39f Add doc values to TermsFacetSearchBenchmark. 2013-10-23 10:58:35 +02:00
Shay Banon a3122a88e4 Java API: Setting track scores does not affect scan search type
When setting track scores, the scan search type will return the scores for each document. The Java API builder does not properly set this value (it only sets it if a sort in in place, which is not relevant for scan search type).

closes #3949
2013-10-22 18:04:22 -07:00
Boaz Leskes 0e6e6f97dc Merge pull request #3940 from rboulton/patch-1
[Docs] Clean up wording in cluster health api doc
2013-10-22 04:09:13 -07:00
Shay Banon 5bc3825c70 externalize writing "raw" fields to helper method 2013-10-21 12:19:14 -07:00
Markus Fischer 782d315da3 Fix markup 2013-10-21 16:11:09 +02:00
Martijn van Groningen 8d49aa398f Added facet support to the percolate api.
Closes #3851
2013-10-21 19:13:28 +07:00
Richard Boulton b62cc7c716 Clean up wording to reduce confusion
The description of the timeout parameter was worded misleadingly; it implied that the API would wait until the cluster reached the desired level and then stayed at that level for the timeout. I've tweaked the sentence to remove the risk of confusion.
2013-10-21 12:37:50 +01:00
Clinton Gormley b2d82d7e75 [DOCS] Reorganised the highlight_query docs and added a version flag 2013-10-18 18:03:31 +02:00