Commit Graph

8665 Commits

Author SHA1 Message Date
Shay Banon 8d793391da [TEST] wait for threads to finish writing before checking
we need to make sure we wait for all threads to finish executing, since there might still be a thread around even post await (i.e. just starting) performing updates
2014-07-06 16:08:09 +02:00
Shay Banon 7ca296987d [TEST] remove repeat lefover... 2014-07-06 12:24:16 +02:00
Shay Banon 6e99448620 [TEST] test didn't take into account other cluster service tasks
The pending tests on an actual node should take into account that other tasks might be executing on that node, thus failing when it happens
2014-07-06 12:23:50 +02:00
Shay Banon 045ce097c9 [TEST] only mark the event as processed when its out of its execution 2014-07-05 19:08:31 +02:00
Britta Weber 2a69ae2f0d Add license header to ManyMappingsBenchmark.java 2014-07-05 18:10:18 +02:00
Shay Banon 1d860f70ca Improve pending api to include current executing class
the pending tasks api will now include the current executing tasks (with a proper marker boolean flag)
this will also help in tests that wait for no pending tasks, to also wait till the current executing task is done
closes #6744
2014-07-05 17:40:52 +02:00
Shay Banon c8e553054b Improve performance for many new fields introduction in mapping
When we have many new fields keep being introduced, the immutable open map we used becomes more and more expensive because of its clone characteristics, and we use it in several places.

The usage semantics of it allows us to also use a CHM if we want to, but it would be nice to still maintain the concurrency aspects of volatile immutable map when the number of fields is sane.

Introduce a new map like data structure, that can switch internally to CHM when a certain threshold is met.

Also add a benchmark class to exploit the many new field mappings use case, which shows significant gains by using this change, to a level where mapping introduction is no longer a significant bottleneck.
closes #6707
2014-07-05 17:39:39 +02:00
Simon Willnauer a9abf18235 [TEST] Close threadpool once test searchcontext is released 2014-07-05 15:20:46 +02:00
Simon Willnauer 10cfc2fa0b [QUERY] Remove Queries#optimizeQuery(...)
This method tires to optimize boolean queries if there is only
one clause. Yet BooleanQuery already does that internally This
optimization is unneeded.

Closes #6743
2014-07-05 13:44:24 +02:00
Shay Banon d5dd9896c8 [TEST] also assert that put mapping is ack'ed 2014-07-05 11:24:16 +02:00
Shay Banon 4baf9df68e [TEST] wait for no pending tasks so processed all mappings 2014-07-05 11:23:10 +02:00
Boaz Leskes 536930c751 [Logging] added trace logging to TransportShardSingleOperationAction
And enabled them on SimpleRecoveryTests.testSimpleRecovery
2014-07-04 21:10:52 +02:00
Clinton Gormley 54790eea10 Update lang-analyzer.asciidoc
Clarified the use of the `stem_exclusion` token filter.

Closes #6613
2014-07-04 17:50:43 +02:00
Shinsuke Sugaya 4bddb4e346 Update plugins.asciidoc 2014-07-05 00:44:02 +09:00
David Pilato f0ad096bc4 Removing plugin does not fail when plugin dir is read only
If you try to remove a plugin in read only dir, you get a successful result:

```
$ bin/plugin --remove marvel
-> Removing marvel
Removed marvel
```

But actually the plugin has not been removed.

When installing, if fails properly:

```
$ bin/plugin -i elasticsearch/marvel/latest
-> Installing elasticsearch/marvel/latest...

Failed to install elasticsearch/marvel/latest, reason: plugin directory /usr/local/elasticsearch/plugins is read only
```

This change throw an exception when we don't succeed removing the plugin.

Closes #6546.
2014-07-04 17:29:18 +02:00
Martijn van Groningen f936283d65 [TEST] Ensure scores are consistent. 2014-07-04 17:18:38 +02:00
Shikhar Bhushan 1e894111b0 Docs: Link to eskka discovery plugin from doc
Closes #6721
2014-07-04 17:06:51 +02:00
Simon Willnauer c4c2796bd9 [TEST] mute test - working on a fix 2014-07-04 16:33:41 +02:00
Chris Earle 312eb2b8b6 PluginManager: tests for missing plugin name when passing --url
Adding code to test for unset plugin names to fail fast with descriptive error messages. Also simplified the series of `if` statements checking for the commands by using a `switch` (now that it's using Java 7), added tests, and updated random exceptions with the up-to-date flag names (e.g., "--verbose" instead of "-verbose").

Closes #5976.
Closes #6013.
2014-07-04 16:18:52 +02:00
Shay Banon 04c573104f make sure we use the new analyzer wrapper on 4.10 2014-07-04 16:06:47 +02:00
Simon Willnauer 1e0506b6a0 [TEST] Reset all cluster if a test hit a failure
Closes #6734
2014-07-04 15:23:12 +02:00
Martijn van Groningen 63852a2c44 Added licence header 2014-07-04 15:11:22 +02:00
Martijn van Groningen 730b83c03c The query string cache can't return the same instance, since Query is mutable changing the query else where in the execution path changes the instance in the cache too.
Instead the query parser cache should return a cloned instances.

Closes #2542
Closes #6733
2014-07-04 15:04:26 +02:00
Simon Willnauer 80617612cb [TEST] Order of the results is not important in this test 2014-07-04 14:29:49 +02:00
Clinton Gormley d3f8c66e26 Updated cache.asciidoc
The index level filter cache was removed a long time ago

Closes #6455
2014-07-04 14:26:20 +02:00
Simon Willnauer 9f5f11c6a2 [TEST] use existing client that is already in the cluster to wait for joining node 2014-07-04 12:18:03 +02:00
Simon Willnauer 1493ece094 [TEST] assert search results before comparing the results - shards / indices might be missing 2014-07-04 11:42:21 +02:00
Simon Willnauer c6623877c9 [Query] QueryParser can return null from a query
This causes a NPE since XContentStructure checks if the query is null
and takes this as the condition to parse from the byte source which is
actually null in that case.

Closes #6722
2014-07-04 10:52:06 +02:00
Simon Willnauer a3d5cdcda8 [TEST] Wait for yellow since some shards might not be started
In this test we only index a handful of docs so if we have more shards
than docs we might fail on the `assertSearchResult` since not all shards
are started but results are just fine.
2014-07-04 09:53:48 +02:00
Shay Banon 5249005578 More resource efficient analysis wrapping usage
Today, we take great care to try and share the same analyzer instances across shards and indices (global analyzer). The idea is to share the same analyzer so the thread local resource it has will not be allocated per analyzer instance per thread.
The problem is that AnalyzerWrapper keeps its resources on its own per thread storage, and with per field reuse strategy, it causes for per field per thread token stream components to be used. This is very evident with the StandardTokenizer that uses a buffer...
This came out of test with "many fields", where the majority of 1GB heap was consumed by StandardTokenizer instances...
closes #6714
2014-07-03 21:03:08 +02:00
Brusic 388fddb3d9 Fix github download link when using specific version 2014-07-03 15:40:16 +02:00
David Pilato 162c62dbcc [DOCS] Add information regarding _type parameter requirement for _mget
Change ID to `[[mget-type]]`

Closes #6670.
2014-07-03 15:38:06 +02:00
David Pilato de48d7f94c [DOCS] Add information regarding _type parameter requirement for _mget
Closes #6670.
2014-07-03 15:23:35 +02:00
Jun Ohtani 0c6a859357 Docs: fixed ICU plugin documentation
add ICU Normalization CharFilter to docs

Closes #6711
2014-07-03 15:21:51 +02:00
Martijn van Groningen 7fbfbabfd3 [TEST] Include mapping in failure 2014-07-03 14:54:20 +02:00
Boaz Leskes ae16956e07 [Discovery] immediately start Master|Node fault detection pinging
After a node joins the clusters, it starts pinging the master to verify it's health. Before, the cluster join request was processed async and we had to give some time to complete. With  #6480 we changed this to wait for the join process to complete on the master. We can therefore start pinging immediately for fast detection of failures. Similar change can be made to the Node fault detection from the master side.

Closes #6706
2014-07-03 14:51:11 +02:00
Simon Willnauer f22e51ae81 [TEST] only call TestCluster#afterTest() if cluster was successfully initialized 2014-07-03 14:42:08 +02:00
Simon Willnauer 0475a052b0 [TEST] disable BWC tests for version < 1.1.0 2014-07-03 14:42:08 +02:00
Simon Willnauer 6d2077b0a3 [TEST] Split up random bulks more often and also if the document set is smallish 2014-07-03 13:56:31 +02:00
Boaz Leskes 7beac4ddbf [Discovery] Fault detection should also check cause exceptions for disconnects
The change introduced in #6686 checks for ConnectionTransportException during pinging. However, transport layer wraps it in  SendRequestTransportException
2014-07-03 13:42:53 +02:00
Mikhail Korobov 955473f475 Docs: unescape regexes in Pattern Tokenizer docs
Currently regexes in Pattern Tokenizer docs are escaped (it seems according to Java rules). I think it is better not to escape them because JSON escaping should be automatic in client libraries, and string escaping depends on a client language used. The default pattern is `\W+`, not `\\W+`.

Closes #6615
2014-07-03 13:34:13 +02:00
hanneskaeufler 6e6f4def5d Docs: Fix typo in timestamp-field.asciidoc
Closes #6661
2014-07-03 13:27:37 +02:00
Simon Willnauer 95b6822f46 [TEST] Exclude SORANI analyzer if compatibility version is < 1.3.0 2014-07-03 13:16:19 +02:00
Robert Muir 2935b751e9 Fix doc formatting. Norwegian stemmers and Scandinavian normalizers
were missing commas between entries.
2014-07-03 07:08:33 -04:00
Simon Willnauer 97793358ea [TEST] Wait for cluster consistency before tests starts 2014-07-03 11:56:05 +02:00
Robert Muir b9a09c2b06 Analysis: Add additional Analyzers, Tokenizers, and TokenFilters from Lucene
Add `irish` analyzer
Add `sorani` analyzer (Kurdish)

Add `classic` tokenizer: specific to english text and tries to recognize hostnames, companies, acronyms, etc.
Add `thai` tokenizer: segments thai text into words.

Add `classic` tokenfilter: cleans up acronyms and possessives from classic tokenizer
Add `apostrophe` tokenfilter: removes text after apostrophe and the apostrophe itself
Add `german_normalization` tokenfilter: umlaut/sharp S normalization
Add `hindi_normalization` tokenfilter: accounts for hindi spelling differences
Add `indic_normalization` tokenfilter: accounts for different unicode representations in Indian languages
Add `sorani_normalization` tokenfilter: normalizes kurdish text
Add `scandinavian_normalization` tokenfilter: normalizes Norwegian, Danish, Swedish text
Add `scandinavian_folding` tokenfilter: much more aggressive form of `scandinavian_normalization`
Add additional languages to stemmer tokenfilter: `galician`, `minimal_galician`, `irish`, `sorani`, `light_nynorsk`, `minimal_nynorsk`

Add support access to default Thai stopword set "_thai_"

Fix some bugs and broken links in documentation.

Closes #5935
2014-07-03 05:47:49 -04:00
Simon Willnauer 9ddfaf3aaf [TEST] Expose `tests.filter` for elasticsearch tests.
`-Dtests.filter` allows to pass filter expressions to the elasticsearch
tests. This allows to filter test annotaged with TestGroup annotations
like @Slow, @Nightly, @Backwards, @Integration with a boolean expresssion like:

 * to run only backwards tests run:
     `mvn -Dtests.bwc.version=X.Y.Z -Dtests.filter="@backwards"`
 * to run all integration tests but skip slow tests run:
     `mvn -Dtests.filter="@integration and not @slow"
 * to take defaults into account ie run all test as well as backwards:
     `mvn -Dtests.filter="default and @backwards"

This feature is a more powerful alternative to flags like
`-Dtests.nighly=true|false` etc.

Closes #6703
2014-07-03 11:40:49 +02:00
Matthew L Daniel 53f2301eea Docs: Add clarifying text about regexp and terms
For the casual reader, the reference to "term queries" may be glossed over, yielding an unexpected result when using `regexp` queries.
This attempts to make that distinction more prominent.

Closes #6698
2014-07-03 11:39:57 +02:00
Simon Willnauer 38e9942bd6 [TEST] Stabelize BWC tests for version < 1.1.0 2014-07-03 11:12:43 +02:00
Boaz Leskes 6e9a1f82b6 [Tests] remove bigArrays which were not fully release from watch list (and still fail the test)
This is to prevent future tests from failing due to these arrays
2014-07-03 11:10:44 +02:00