Commit Graph

5354 Commits

Author SHA1 Message Date
Simon Willnauer ba13930b32 Fix test include pattern to include *Test.class
We missed *Test.class which is not our convention but we could miss
some tests. We should better include the *Test.class tests as well.
2013-08-13 17:46:48 +02:00
Simon Willnauer 7e1d8a6ca3 Raise default DeleteIndex Timeout
Currently the timeout for an delete index operation is set to 10 seconds.
Yet, if a full flush is running while we delete and index this can
easily exceed 10 seconds. The timeout is not dramatic ie. the index
will be deleted eventually but the client request is not acked which
can cause confusion. We should raise it to prevent unnecessary confusion
especially in client tests where this can happen if the machine is pretty busy.

The new timeout is set to 60 seconds.

Closes #3498
2013-08-13 17:28:19 +02:00
Shay Banon 534299a27c function score test cleanup
- also, properly report on the failed assertion in toFloat
- use function score in the explain compared to custom score
- use the Tests suffix convention
2013-08-13 17:23:38 +02:00
Boaz Leskes 9d28002077 UpdateMappingTests - updateDefaultMappingSettings now creates the index with a mapping. 2013-08-13 17:16:11 +02:00
Boaz Leskes e5f459af83 Allow to update the _source mapping exclude/include dynamically when we merge mappings.
Closes #3491
2013-08-13 17:09:02 +02:00
David Pilato 328608f55f Make RestSearchAction#parseSearchXXX(RestRequest) public
When building a plugin with a new search endpoint, you need to parse the request as a searchRequest.

Methods exist in RestSearchAction class but are private.

We will modify them to be public static. This applies to:

* `RestSearchAction#parseSearchRequest(RestRequest)`
* `RestSearchAction#parseSearchSource(RestRequest)`

 Closes #3499.
2013-08-13 17:06:04 +02:00
Shay Banon e111a7da62 lazily create the no shard available exception 2013-08-13 16:41:16 +02:00
Boaz Leskes acf17b4e39 Added some comments regarding the acknowledgement logic in MetaDataMappingService.putMapping
Made left over cluster state debug log entry less verbose.
2013-08-13 14:32:51 +02:00
Martijn van Groningen e8909396b4 Removed todo 2013-08-13 14:26:19 +02:00
Shay Banon af17ae55ab remove the assert on AnalyzerWrapper
see https://issues.apache.org/jira/browse/LUCENE-5170
2013-08-13 12:15:05 +02:00
Shay Banon 9126d11824 better log message for none gateway, also make it debug level 2013-08-13 00:19:50 +02:00
Simon Willnauer c6a803b677 Also catch EsRejectedExecutionException next to
RejectedExcecutionException
2013-08-12 21:25:40 +02:00
Martijn van Groningen bc0abd8226 Added multi percolate api
The multi percolate allows the bundle multiple percolate requests into one request. This api works similar to the multi search api. The request body format is line based. Each percolate request item takes two lines, the first line is the header and the second line is the body.

The header can contain any parameter that normally would be set via the request path or query string parameters. There are several percolate actions, because there are multiple types of percolate requests:
* `percolate` - Action for defining a regular percolate request.
* `count_percolate` - Action for defining a count percolate request.
* `percolate_existing_doc` - Action for defining a percolate existing document request.
* `count_percolate_existing_doc` - Action for defining a count percolate existing document request.

Each action has its own set of parameters that need to be specified in the percolate action.
Format:
```
{"[header_type]" : {[options...]}
{[body]}
```

Depending on the percolate action different parameters can be specified. For example the percolate and percolate existing document actions support different parameters.

The following endpoints are supported:
```
POST localhost:9200/[index]/[type]/_mpercolate
POST localhost:9200/[index]/_mpercolate
POST localhost:9200/_mpercolate
```

The `index` and `type` defined in the url path are the default index and type.

Closes #3488
2013-08-12 18:32:28 +02:00
Simon Willnauer 82d3693a91 Catch Throwable rather than Exception if latches are present. 2013-08-12 17:46:44 +02:00
Simon Willnauer 8a876ea80e Limit the number of extracted token instance per query token.
FVH deploys some recursive logic to extract terms from documents
that need to highlighted. For documents that have terms with super
large term frequency like a document that repeats a terms very
very often this can produce some very large stacks when extracting
the terms. Taken to an extreme this causes stack overflow errors
when this grow beyond a term frequency >= 6000.

The ultimate solution is a iterative implementation of the extract
logic but until then we should protect users from these massive
term extractions which might be not very useful in the first place.

Closes #3486
2013-08-12 17:46:44 +02:00
Boaz Leskes ab6163898f Postponed acknowledging put mapping requests to after master has finished processed them
Also - TransportMasterNodeOperationAction was potentially use stale cluster state

Closes #3487
2013-08-12 17:00:47 +02:00
Martijn van Groningen 4b25e6b63e Changed default operation_threading from single_thread to thread_per_shard.
Closes #3483
2013-08-12 15:30:09 +02:00
Simon Willnauer 8a48e2f969 Use awaitBusy rather than hand crafted version in tests. 2013-08-12 15:17:48 +02:00
Alexander Reelsen f58f165522 Support fuzzy queries in CompletionSuggest
Added the FuzzySuggester in order to support completion queries

The following options have been added for the fuxxy suggester

* edit_distance: Maximum edit distance
* transpositions: Sets if transpositions should be counted as one or two changes
* min_prefix_len: Minimum length of the input before fuzzy suggestions are returned
* non_prefix_len: Minimum length of the input, which is not checked for fuzzy alternatives

Closes #3465
2013-08-12 15:07:07 +02:00
Alexander Reelsen a7b643305a Fix debian init script to not depend on new start-stop-daemon
By making use of the lsb provided functions, one does not depend on the start-stop daemon version to test if elasticsearch is running.
This ensures, that the init script works on debian wheezy, squeeze, current ubuntu and LTS versions.

Closes #3452
2013-08-12 15:03:42 +02:00
Alexander Reelsen 5c853fb22d Use TransportMasterNodeOperationAction in TransportGetIndexTemplatesAction
No need to use ClusterInfoRequest, as we do not need to access any indices.
2013-08-12 14:48:20 +02:00
Martijn van Groningen 4d40a1e77c Set default `operation_threading` to `thread_per_shard` and exposed it as an option in the rest api. 2013-08-12 14:45:51 +02:00
Simon Willnauer 59be83f9fc Remove accidentially committed default values
`-Dtests.maxFailures` and `-Dtests.failfast` should not be enabled
by default.
2013-08-12 14:32:04 +02:00
Alexander Reelsen 45c0d1de04 Added IndicesAdminClient.getIndexTemplates()
In addition to creating and removing a template, one can now receive index templates as well.
Simple regexes like template* are supported.

Closes #3439
2013-08-12 13:56:25 +02:00
Simon Willnauer dbed36a13f Added support for `tests.failfast` and `tests.maxiters`
This commit adds support for failing fast when running a test
case with `-Dtests.iters=N` and uses some goodness from LuceneTestCase
in a new base `AbstractRandomizedTest`. This class checks among other
things if a tests doesn't call `super.setup` / `super.tearDown` when it
should do and checks if a large static resources are not cleaned up
after the tests ie. a running node.
2013-08-12 13:18:56 +02:00
Martijn van Groningen 2ed3dbbf67 Test deleting the percolate type differently. Instead of checking the types exist api, register a DocumentTypeListener that notifies when percolate queries have been cleared. 2013-08-12 12:05:22 +02:00
David Pilato 83c26eb74a NPE for POST mode facets if facet_filter gives no document.
Closes #3479.
2013-08-11 21:21:02 +02:00
Shay Banon 5c7d7fb399 Failure to execute search request with empty top level filter
closes #3477
2013-08-10 10:21:30 +02:00
Simon Willnauer be103c188b Disable UpdateMappingTets#updateDefaultMappingSettings
Test has been too flaky over nightly builds. Disabling it
with AwaitFix.
2013-08-10 07:57:58 +02:00
Boaz Leskes 4debf44cd9 Separated index creation from mapping creation pending bug fix concerning concurrent not-acked mapping requests 2013-08-09 21:39:47 +02:00
Boaz Leskes 5f4dc5433e when changing the mapping of the _default_ mapping, do not apply the old _default_ mapping to the new one and also do not validate the new version with a merge but parse is as a new type.
Closes #3474, Closes #3476
2013-08-09 20:15:51 +02:00
Britta Weber f64065c9d2 termvectors: fix null pointer exception if field has no term vectors
Retrieving termvectors for a document that does not have the requested field
caused a null pointer exception. Same for documents if the field has no term vectors,
for example, because the field only contains "?".
Now, an empty response is returned.

Closes #3471
2013-08-09 15:06:09 +02:00
Simon Willnauer ec770373ab Added random sort test for dense and sparse fields.
This test triggers a MultiDoc / MultiOrds in-memory representation
even if the field is not multivalued

Relates to #3470
2013-08-09 14:15:26 +02:00
Simon Willnauer 417c193cc3 Return ordinals from MultiOrdinals.MultiDocs
MultiOrdinals.MultiDocs returned 'null' ordinals which caused
a NPE if the field was single valued and would allow a significantly
smaller in memory representation than single packed int ordinals.

Closes #3470
2013-08-09 08:03:08 +02:00
Simon Willnauer 2ed87b5312 Use nonzero status code to signal abnormal termination
We currently return with status code 0 when an IOException occurs.
The plugin manager should in any case return a nonzero status if
the operation was not successful. Now the PluginManager uses the
following reponse codes based on 'sysexists.sh':
 * '0' on success
 * '64' command line usage error
 * '70' internal software error
 * '74' input/output

Closes #3463
2013-08-08 17:48:56 +02:00
Martijn van Groningen f8f8cac0ed ttl can be as lower than 0 (purge interval) 2013-08-08 17:43:11 +02:00
Martijn van Groningen c568fb6344 In case ttl has passed, then just check the delete count 2013-08-08 17:42:12 +02:00
Simon Willnauer 5b8ce393db Create mapping ahead of time and don't rely on index request in test 2013-08-08 17:28:59 +02:00
Simon Willnauer 4e2b9ff2ad Expose 'index.compound_on_flush' via engine settings
Lucene 4.4 shipped with a fundamental change in how the decision
on when to write compound files is made. During segment flush the
compound files are written by default which solely relies on a flag
in the IndexWriterConfig. The merge policy has been factored out to
only make decisions on merges and not on IW flushes. The default now
is always writing CFS on flush to reduce resource usage like open files
etc. if segments are flushed regularly. While providing a senseable
default certain users / usecases might need to change this setting if
re-packing flushed segments into CFS is not desired.

Closes #3461
2013-08-08 13:36:05 +02:00
Simon Willnauer 04b23a8fab Catch RejectedExecutionException on node shutdown 2013-08-08 13:10:13 +02:00
Simon Willnauer ef365098e7 Use DiscoveryModule instead of ClusterService to obtain local node id
The ClusterService might not see the latest cluster state and therefore
might not contain the local node id. Discovery will always see the local
node id since it's set on startup.
2013-08-08 12:39:49 +02:00
Martijn van Groningen d450d3b016 Simplified checks 2013-08-08 11:33:06 +02:00
Shay Banon c7d5881686 make sure we add the _uid as the first field in a doc
this will improve early termination loading times, but requires potential improvements in Lucene in terms of decompression
2013-08-07 23:28:07 +02:00
Simon Willnauer 6c91ff83f2 Assert on index delete in tests to ensure all indices are wiped even on disk 2013-08-07 17:56:55 +02:00
Simon Willnauer bcda6dfe54 Remove random empty string from test since it triggers a different exception 2013-08-07 14:17:27 +02:00
Shay Banon 80fa91d873 improve effort into figuring out the shard associated with a search failure 2013-08-07 14:16:22 +02:00
Martijn van Groningen d26b165af3 Added improvements for terms filter on _parent field similar to what has been for term filter.
Relates to #3454
2013-08-07 14:05:40 +02:00
Simon Willnauer f2dc4f810c Added tests for malformed mappings with no root object
This commit also makes the error message more consistent with
other exception messages in the DocumentMapperParser.
2013-08-07 14:01:32 +02:00
Manuel Bernhardt 27518b5e41 Improved error message when the mapping document is malformed 2013-08-07 13:41:49 +02:00
Simon Willnauer 7f0115ba9a Return nothing instead of everything in MLT if no field is supported.
Today due the optimizations in the boolean query builder we adjust
a pure negative query with a 'match_all'. This is not the desired
behavior in the MLT API if all the fields in a document are unsupported.
If that happens today we return all documents but the one MLT is
executed on.

Closes #3453
2013-08-07 13:25:09 +02:00