Commit Graph

8730 Commits

Author SHA1 Message Date
Boaz Leskes 83770c2583 [Test] Revert temporary fixes for update mapping on recovery issue
This reverts:
"Test: Temporarily change delete/put_mapping to wait for green": commit e408f8f638c2dd97a3ec86c8a9ac940f43ab37a0.
"[TEST] wait for green to update mapping": commit b3641a2ee6eb23318d49f5f04b39149e70c2b65b.
2014-07-07 22:27:25 +02:00
Boaz Leskes 867d88795b [Recovery] only send mapping updates to master if needed
The change added in #6762 helps making sure the pending mapping updates are processed on all nodes to prevent moving shards to nodes which are not yet fully aware of the new mapping. However it introduced a racing condition delete_mapping operations, potentially causing a type to be added after it's deletion. This commit solves this by only sending a mapping update if the mapping source has actually changed.

Closes #6772
2014-07-07 21:39:22 +02:00
Clinton Gormley 34893c0570 Test: Temporarily change delete/put_mapping to wait for green
Delete and update mapping execution order has changed with #6762
2014-07-07 19:02:36 +02:00
Shay Banon 135404fffc [TEST] wait for green to update mapping
before we update the mapping, make sure the index is green, so a delayed recovery update mapping will not override the mapping itself
2014-07-07 18:42:20 +02:00
Shay Banon 807b2409ee Don't update default mapping on phase2 recovery
the default mapping is not merged, but updated in place, and only put mapping API can change it, no need to make sure it has been properly updated on the master. This can cause conflicts when a put mapping for it happens at the same time.
closes #6771
2014-07-07 18:40:05 +02:00
mikemccand 15b81c91f7 Core: remove per-ID locking when ID was auto-generated
When we know the ID for the document we are about to index was
auto-generated, we don't need to acquire/release the per-ID lock,
which might provide small speedups during highly concurrent indexing.

Closes #6584
2014-07-07 11:35:33 -04:00
Alexander Reelsen ec92646289 Build: Add all netty classes during shading
In order to have access to all codecs and handlers by netty, they
need to be exposed during shading, otherwise only the classes, which
are used by the built are exposed.
2014-07-07 15:09:36 +02:00
Clinton Gormley f97ff704cf Query DSL: Improved explanation for match_phrase_prefix
The match_phrase_prefix provided the same explanation as the match_phrase
query. There was no indication that the last term was run as a prefix
query.

This change marks the last term (or terms if there are multiple terms
in the same position) with a *

Closes #2449
2014-07-07 14:42:31 +02:00
Boaz Leskes 7f5f4e842e [Relocation] process pending mapping update in phase 2
During phase1 we copy over all lucene segments. These may refer to mapping updates that are still queued up to be sent to master. We must make sure those pending updates are processed before completing the relocation.

Relates to #6648

Closes #6762
2014-07-07 13:21:16 +02:00
Lee Hinman 7023caa1a1 Allow primaries that have never been allocated to be allocated if under the low watermark
Fixes #6196
2014-07-07 11:48:11 +02:00
Andrii Gakhov 80321d89d9 Docs: Update histogram-aggregation.asciidoc
filter in a filtered query should be under "filter" key

Closes #6738
2014-07-07 10:44:11 +02:00
Carsten Brandt bd4699da7e Docs: fixed a typo in the docs
Closes: #6718
2014-07-07 10:41:36 +02:00
Clinton Gormley e4baa56f4b Docs: Language analyzers
Clarified the use of stem_exclusion and the keyword_marker
token filter

Closes #6613
2014-07-07 10:06:18 +02:00
Shay Banon 5093f050ab Upgrade Jackson to 2.4.1
closes #6757
2014-07-07 09:49:53 +02:00
Shay Banon b471aeb24c Call callback on actual mapping processed
only callback the registered callback listeners when mapping have actually been processed...
closes #6748
2014-07-07 09:42:05 +02:00
Shay Banon 8d793391da [TEST] wait for threads to finish writing before checking
we need to make sure we wait for all threads to finish executing, since there might still be a thread around even post await (i.e. just starting) performing updates
2014-07-06 16:08:09 +02:00
Shay Banon 7ca296987d [TEST] remove repeat lefover... 2014-07-06 12:24:16 +02:00
Shay Banon 6e99448620 [TEST] test didn't take into account other cluster service tasks
The pending tests on an actual node should take into account that other tasks might be executing on that node, thus failing when it happens
2014-07-06 12:23:50 +02:00
Shay Banon 045ce097c9 [TEST] only mark the event as processed when its out of its execution 2014-07-05 19:08:31 +02:00
Britta Weber 2a69ae2f0d Add license header to ManyMappingsBenchmark.java 2014-07-05 18:10:18 +02:00
Shay Banon 1d860f70ca Improve pending api to include current executing class
the pending tasks api will now include the current executing tasks (with a proper marker boolean flag)
this will also help in tests that wait for no pending tasks, to also wait till the current executing task is done
closes #6744
2014-07-05 17:40:52 +02:00
Shay Banon c8e553054b Improve performance for many new fields introduction in mapping
When we have many new fields keep being introduced, the immutable open map we used becomes more and more expensive because of its clone characteristics, and we use it in several places.

The usage semantics of it allows us to also use a CHM if we want to, but it would be nice to still maintain the concurrency aspects of volatile immutable map when the number of fields is sane.

Introduce a new map like data structure, that can switch internally to CHM when a certain threshold is met.

Also add a benchmark class to exploit the many new field mappings use case, which shows significant gains by using this change, to a level where mapping introduction is no longer a significant bottleneck.
closes #6707
2014-07-05 17:39:39 +02:00
Simon Willnauer a9abf18235 [TEST] Close threadpool once test searchcontext is released 2014-07-05 15:20:46 +02:00
Simon Willnauer 10cfc2fa0b [QUERY] Remove Queries#optimizeQuery(...)
This method tires to optimize boolean queries if there is only
one clause. Yet BooleanQuery already does that internally This
optimization is unneeded.

Closes #6743
2014-07-05 13:44:24 +02:00
Shay Banon d5dd9896c8 [TEST] also assert that put mapping is ack'ed 2014-07-05 11:24:16 +02:00
Shay Banon 4baf9df68e [TEST] wait for no pending tasks so processed all mappings 2014-07-05 11:23:10 +02:00
Boaz Leskes 536930c751 [Logging] added trace logging to TransportShardSingleOperationAction
And enabled them on SimpleRecoveryTests.testSimpleRecovery
2014-07-04 21:10:52 +02:00
Clinton Gormley 54790eea10 Update lang-analyzer.asciidoc
Clarified the use of the `stem_exclusion` token filter.

Closes #6613
2014-07-04 17:50:43 +02:00
Shinsuke Sugaya 4bddb4e346 Update plugins.asciidoc 2014-07-05 00:44:02 +09:00
David Pilato f0ad096bc4 Removing plugin does not fail when plugin dir is read only
If you try to remove a plugin in read only dir, you get a successful result:

```
$ bin/plugin --remove marvel
-> Removing marvel
Removed marvel
```

But actually the plugin has not been removed.

When installing, if fails properly:

```
$ bin/plugin -i elasticsearch/marvel/latest
-> Installing elasticsearch/marvel/latest...

Failed to install elasticsearch/marvel/latest, reason: plugin directory /usr/local/elasticsearch/plugins is read only
```

This change throw an exception when we don't succeed removing the plugin.

Closes #6546.
2014-07-04 17:29:18 +02:00
Martijn van Groningen f936283d65 [TEST] Ensure scores are consistent. 2014-07-04 17:18:38 +02:00
Shikhar Bhushan 1e894111b0 Docs: Link to eskka discovery plugin from doc
Closes #6721
2014-07-04 17:06:51 +02:00
Simon Willnauer c4c2796bd9 [TEST] mute test - working on a fix 2014-07-04 16:33:41 +02:00
Chris Earle 312eb2b8b6 PluginManager: tests for missing plugin name when passing --url
Adding code to test for unset plugin names to fail fast with descriptive error messages. Also simplified the series of `if` statements checking for the commands by using a `switch` (now that it's using Java 7), added tests, and updated random exceptions with the up-to-date flag names (e.g., "--verbose" instead of "-verbose").

Closes #5976.
Closes #6013.
2014-07-04 16:18:52 +02:00
Shay Banon 04c573104f make sure we use the new analyzer wrapper on 4.10 2014-07-04 16:06:47 +02:00
Simon Willnauer 1e0506b6a0 [TEST] Reset all cluster if a test hit a failure
Closes #6734
2014-07-04 15:23:12 +02:00
Martijn van Groningen 63852a2c44 Added licence header 2014-07-04 15:11:22 +02:00
Martijn van Groningen 730b83c03c The query string cache can't return the same instance, since Query is mutable changing the query else where in the execution path changes the instance in the cache too.
Instead the query parser cache should return a cloned instances.

Closes #2542
Closes #6733
2014-07-04 15:04:26 +02:00
Simon Willnauer 80617612cb [TEST] Order of the results is not important in this test 2014-07-04 14:29:49 +02:00
Clinton Gormley d3f8c66e26 Updated cache.asciidoc
The index level filter cache was removed a long time ago

Closes #6455
2014-07-04 14:26:20 +02:00
Simon Willnauer 9f5f11c6a2 [TEST] use existing client that is already in the cluster to wait for joining node 2014-07-04 12:18:03 +02:00
Simon Willnauer 1493ece094 [TEST] assert search results before comparing the results - shards / indices might be missing 2014-07-04 11:42:21 +02:00
Simon Willnauer c6623877c9 [Query] QueryParser can return null from a query
This causes a NPE since XContentStructure checks if the query is null
and takes this as the condition to parse from the byte source which is
actually null in that case.

Closes #6722
2014-07-04 10:52:06 +02:00
Simon Willnauer a3d5cdcda8 [TEST] Wait for yellow since some shards might not be started
In this test we only index a handful of docs so if we have more shards
than docs we might fail on the `assertSearchResult` since not all shards
are started but results are just fine.
2014-07-04 09:53:48 +02:00
Shay Banon 5249005578 More resource efficient analysis wrapping usage
Today, we take great care to try and share the same analyzer instances across shards and indices (global analyzer). The idea is to share the same analyzer so the thread local resource it has will not be allocated per analyzer instance per thread.
The problem is that AnalyzerWrapper keeps its resources on its own per thread storage, and with per field reuse strategy, it causes for per field per thread token stream components to be used. This is very evident with the StandardTokenizer that uses a buffer...
This came out of test with "many fields", where the majority of 1GB heap was consumed by StandardTokenizer instances...
closes #6714
2014-07-03 21:03:08 +02:00
Brusic 388fddb3d9 Fix github download link when using specific version 2014-07-03 15:40:16 +02:00
David Pilato 162c62dbcc [DOCS] Add information regarding _type parameter requirement for _mget
Change ID to `[[mget-type]]`

Closes #6670.
2014-07-03 15:38:06 +02:00
David Pilato de48d7f94c [DOCS] Add information regarding _type parameter requirement for _mget
Closes #6670.
2014-07-03 15:23:35 +02:00
Jun Ohtani 0c6a859357 Docs: fixed ICU plugin documentation
add ICU Normalization CharFilter to docs

Closes #6711
2014-07-03 15:21:51 +02:00
Martijn van Groningen 7fbfbabfd3 [TEST] Include mapping in failure 2014-07-03 14:54:20 +02:00