Commit Graph

6853 Commits

Author SHA1 Message Date
Britta Weber ebbd00acc2 Fix some minor things in function score parser/builder
- remove default scale weight in builder
- make parameters object/double instead of string
- do not convert number to string and back again, parse double instead
- remove javadoc reference to test classes
- Set parameters in constructor instead of in method
2013-08-14 14:21:07 +02:00
Britta Weber 592e637293 remove check and test for more than one mapper per field 2013-08-14 14:21:07 +02:00
Martijn van Groningen 691ac8e105 Added scoring support to percolate api
Scoring support will allow the percolate matches to be sorted, or just assign a scores to percolate matches. Sorting by score can be very useful when millions of matching percolate queries are being returned.

The scoring support hooks in into the percolate query option and adds two new boolean options:
* `sort` - Whether to sort the matches based on the score. This will also include the score for each match. The `size` option is a required option when sorting percolate matches is enabled.
* `score` - Whether to compute the score and include it with each match. This will not sort the matches.

For both new options the `query` option needs to be specified, which is used to produce the scores. The `query` option is normally used to control which percolate queries are evaluated. In order to give meaning to these scores, the recently added `function_score` query in #3423 can be used to wrap the percolate query, this way the scores have meaning.

Closes #3506
2013-08-14 13:51:13 +02:00
Britta Weber 32cdddb671 remove sysout 2013-08-14 10:38:02 +02:00
Shay Banon 2f1680839f empty double/long values should return 0
to conform with all other implementations (non empty), they getValue when there is no value associated with a doc should be 0
2013-08-14 00:06:40 +02:00
Shay Banon eb9c0d077b no need doc action test to check count in before class
- also, since we randomize client transports, no need for specific classes to test for it, we test different clients across all our tests
2013-08-14 00:02:29 +02:00
Shay Banon 3db8be6c77 rename class to conform with Tests suffix 2013-08-13 20:38:57 +02:00
Shay Banon f1467dbde2 empty numeric field data should retain the correct num docs
the fact that there are no values in the numeric field data, doesn't mean there are no docs, behavior should be to the bytes variant
2013-08-13 18:52:35 +02:00
Simon Willnauer 8774c46cc5 Fix assert to check the deviation rather than the absolute difference.
Deviation should be less or equal to 0.01 ~ 1% after the cast.
2013-08-13 17:46:57 +02:00
Simon Willnauer ba13930b32 Fix test include pattern to include *Test.class
We missed *Test.class which is not our convention but we could miss
some tests. We should better include the *Test.class tests as well.
2013-08-13 17:46:48 +02:00
Simon Willnauer 7e1d8a6ca3 Raise default DeleteIndex Timeout
Currently the timeout for an delete index operation is set to 10 seconds.
Yet, if a full flush is running while we delete and index this can
easily exceed 10 seconds. The timeout is not dramatic ie. the index
will be deleted eventually but the client request is not acked which
can cause confusion. We should raise it to prevent unnecessary confusion
especially in client tests where this can happen if the machine is pretty busy.

The new timeout is set to 60 seconds.

Closes #3498
2013-08-13 17:28:19 +02:00
Shay Banon 534299a27c function score test cleanup
- also, properly report on the failed assertion in toFloat
- use function score in the explain compared to custom score
- use the Tests suffix convention
2013-08-13 17:23:38 +02:00
Boaz Leskes 9d28002077 UpdateMappingTests - updateDefaultMappingSettings now creates the index with a mapping. 2013-08-13 17:16:11 +02:00
Boaz Leskes e5f459af83 Allow to update the _source mapping exclude/include dynamically when we merge mappings.
Closes #3491
2013-08-13 17:09:02 +02:00
David Pilato 328608f55f Make RestSearchAction#parseSearchXXX(RestRequest) public
When building a plugin with a new search endpoint, you need to parse the request as a searchRequest.

Methods exist in RestSearchAction class but are private.

We will modify them to be public static. This applies to:

* `RestSearchAction#parseSearchRequest(RestRequest)`
* `RestSearchAction#parseSearchSource(RestRequest)`

 Closes #3499.
2013-08-13 17:06:04 +02:00
Shay Banon e111a7da62 lazily create the no shard available exception 2013-08-13 16:41:16 +02:00
Boaz Leskes acf17b4e39 Added some comments regarding the acknowledgement logic in MetaDataMappingService.putMapping
Made left over cluster state debug log entry less verbose.
2013-08-13 14:32:51 +02:00
Martijn van Groningen e8909396b4 Removed todo 2013-08-13 14:26:19 +02:00
Shay Banon af17ae55ab remove the assert on AnalyzerWrapper
see https://issues.apache.org/jira/browse/LUCENE-5170
2013-08-13 12:15:05 +02:00
Shay Banon 9126d11824 better log message for none gateway, also make it debug level 2013-08-13 00:19:50 +02:00
Simon Willnauer c6a803b677 Also catch EsRejectedExecutionException next to
RejectedExcecutionException
2013-08-12 21:25:40 +02:00
Martijn van Groningen bc0abd8226 Added multi percolate api
The multi percolate allows the bundle multiple percolate requests into one request. This api works similar to the multi search api. The request body format is line based. Each percolate request item takes two lines, the first line is the header and the second line is the body.

The header can contain any parameter that normally would be set via the request path or query string parameters. There are several percolate actions, because there are multiple types of percolate requests:
* `percolate` - Action for defining a regular percolate request.
* `count_percolate` - Action for defining a count percolate request.
* `percolate_existing_doc` - Action for defining a percolate existing document request.
* `count_percolate_existing_doc` - Action for defining a count percolate existing document request.

Each action has its own set of parameters that need to be specified in the percolate action.
Format:
```
{"[header_type]" : {[options...]}
{[body]}
```

Depending on the percolate action different parameters can be specified. For example the percolate and percolate existing document actions support different parameters.

The following endpoints are supported:
```
POST localhost:9200/[index]/[type]/_mpercolate
POST localhost:9200/[index]/_mpercolate
POST localhost:9200/_mpercolate
```

The `index` and `type` defined in the url path are the default index and type.

Closes #3488
2013-08-12 18:32:28 +02:00
Simon Willnauer 82d3693a91 Catch Throwable rather than Exception if latches are present. 2013-08-12 17:46:44 +02:00
Simon Willnauer 8a876ea80e Limit the number of extracted token instance per query token.
FVH deploys some recursive logic to extract terms from documents
that need to highlighted. For documents that have terms with super
large term frequency like a document that repeats a terms very
very often this can produce some very large stacks when extracting
the terms. Taken to an extreme this causes stack overflow errors
when this grow beyond a term frequency >= 6000.

The ultimate solution is a iterative implementation of the extract
logic but until then we should protect users from these massive
term extractions which might be not very useful in the first place.

Closes #3486
2013-08-12 17:46:44 +02:00
Boaz Leskes ab6163898f Postponed acknowledging put mapping requests to after master has finished processed them
Also - TransportMasterNodeOperationAction was potentially use stale cluster state

Closes #3487
2013-08-12 17:00:47 +02:00
Martijn van Groningen 4b25e6b63e Changed default operation_threading from single_thread to thread_per_shard.
Closes #3483
2013-08-12 15:30:09 +02:00
Simon Willnauer 8a48e2f969 Use awaitBusy rather than hand crafted version in tests. 2013-08-12 15:17:48 +02:00
Alexander Reelsen f58f165522 Support fuzzy queries in CompletionSuggest
Added the FuzzySuggester in order to support completion queries

The following options have been added for the fuxxy suggester

* edit_distance: Maximum edit distance
* transpositions: Sets if transpositions should be counted as one or two changes
* min_prefix_len: Minimum length of the input before fuzzy suggestions are returned
* non_prefix_len: Minimum length of the input, which is not checked for fuzzy alternatives

Closes #3465
2013-08-12 15:07:07 +02:00
Alexander Reelsen a7b643305a Fix debian init script to not depend on new start-stop-daemon
By making use of the lsb provided functions, one does not depend on the start-stop daemon version to test if elasticsearch is running.
This ensures, that the init script works on debian wheezy, squeeze, current ubuntu and LTS versions.

Closes #3452
2013-08-12 15:03:42 +02:00
Alexander Reelsen 5c853fb22d Use TransportMasterNodeOperationAction in TransportGetIndexTemplatesAction
No need to use ClusterInfoRequest, as we do not need to access any indices.
2013-08-12 14:48:20 +02:00
Martijn van Groningen 4d40a1e77c Set default `operation_threading` to `thread_per_shard` and exposed it as an option in the rest api. 2013-08-12 14:45:51 +02:00
Simon Willnauer 59be83f9fc Remove accidentially committed default values
`-Dtests.maxFailures` and `-Dtests.failfast` should not be enabled
by default.
2013-08-12 14:32:04 +02:00
Alexander Reelsen 45c0d1de04 Added IndicesAdminClient.getIndexTemplates()
In addition to creating and removing a template, one can now receive index templates as well.
Simple regexes like template* are supported.

Closes #3439
2013-08-12 13:56:25 +02:00
Simon Willnauer dbed36a13f Added support for `tests.failfast` and `tests.maxiters`
This commit adds support for failing fast when running a test
case with `-Dtests.iters=N` and uses some goodness from LuceneTestCase
in a new base `AbstractRandomizedTest`. This class checks among other
things if a tests doesn't call `super.setup` / `super.tearDown` when it
should do and checks if a large static resources are not cleaned up
after the tests ie. a running node.
2013-08-12 13:18:56 +02:00
Martijn van Groningen 2ed3dbbf67 Test deleting the percolate type differently. Instead of checking the types exist api, register a DocumentTypeListener that notifies when percolate queries have been cleared. 2013-08-12 12:05:22 +02:00
David Pilato 83c26eb74a NPE for POST mode facets if facet_filter gives no document.
Closes #3479.
2013-08-11 21:21:02 +02:00
Shay Banon 5c7d7fb399 Failure to execute search request with empty top level filter
closes #3477
2013-08-10 10:21:30 +02:00
Simon Willnauer be103c188b Disable UpdateMappingTets#updateDefaultMappingSettings
Test has been too flaky over nightly builds. Disabling it
with AwaitFix.
2013-08-10 07:57:58 +02:00
Boaz Leskes 4debf44cd9 Separated index creation from mapping creation pending bug fix concerning concurrent not-acked mapping requests 2013-08-09 21:39:47 +02:00
Boaz Leskes 5f4dc5433e when changing the mapping of the _default_ mapping, do not apply the old _default_ mapping to the new one and also do not validate the new version with a merge but parse is as a new type.
Closes #3474, Closes #3476
2013-08-09 20:15:51 +02:00
Britta Weber f64065c9d2 termvectors: fix null pointer exception if field has no term vectors
Retrieving termvectors for a document that does not have the requested field
caused a null pointer exception. Same for documents if the field has no term vectors,
for example, because the field only contains "?".
Now, an empty response is returned.

Closes #3471
2013-08-09 15:06:09 +02:00
Simon Willnauer ec770373ab Added random sort test for dense and sparse fields.
This test triggers a MultiDoc / MultiOrds in-memory representation
even if the field is not multivalued

Relates to #3470
2013-08-09 14:15:26 +02:00
Simon Willnauer 417c193cc3 Return ordinals from MultiOrdinals.MultiDocs
MultiOrdinals.MultiDocs returned 'null' ordinals which caused
a NPE if the field was single valued and would allow a significantly
smaller in memory representation than single packed int ordinals.

Closes #3470
2013-08-09 08:03:08 +02:00
Simon Willnauer 2ed87b5312 Use nonzero status code to signal abnormal termination
We currently return with status code 0 when an IOException occurs.
The plugin manager should in any case return a nonzero status if
the operation was not successful. Now the PluginManager uses the
following reponse codes based on 'sysexists.sh':
 * '0' on success
 * '64' command line usage error
 * '70' internal software error
 * '74' input/output

Closes #3463
2013-08-08 17:48:56 +02:00
Martijn van Groningen f8f8cac0ed ttl can be as lower than 0 (purge interval) 2013-08-08 17:43:11 +02:00
Martijn van Groningen c568fb6344 In case ttl has passed, then just check the delete count 2013-08-08 17:42:12 +02:00
Simon Willnauer 5b8ce393db Create mapping ahead of time and don't rely on index request in test 2013-08-08 17:28:59 +02:00
Simon Willnauer 4e2b9ff2ad Expose 'index.compound_on_flush' via engine settings
Lucene 4.4 shipped with a fundamental change in how the decision
on when to write compound files is made. During segment flush the
compound files are written by default which solely relies on a flag
in the IndexWriterConfig. The merge policy has been factored out to
only make decisions on merges and not on IW flushes. The default now
is always writing CFS on flush to reduce resource usage like open files
etc. if segments are flushed regularly. While providing a senseable
default certain users / usecases might need to change this setting if
re-packing flushed segments into CFS is not desired.

Closes #3461
2013-08-08 13:36:05 +02:00
Simon Willnauer 04b23a8fab Catch RejectedExecutionException on node shutdown 2013-08-08 13:10:13 +02:00
Simon Willnauer ef365098e7 Use DiscoveryModule instead of ClusterService to obtain local node id
The ClusterService might not see the latest cluster state and therefore
might not contain the local node id. Discovery will always see the local
node id since it's set on startup.
2013-08-08 12:39:49 +02:00