We check if the version map needs to be refreshed after we released
the readlock which can cause the the engine being closed before we
read the value from the volatile `indexWriter` field which can cause an
NPE on the indexing thread. This commit also fixes a potential uncaught
exception if the refresh failed due to the engine being already closed.
Relates to #6443Closes#6786
remove internal callas on FieldMappers#Names, we properly reuse FieldMapper, so there is no need to try and call intern in order to reuse the names. This can be heavy with many fields and continuous mapping parsing.
closes#6747
We use awaitBusy in our tests, the problem is that we have to check if it awaited or not, and then try and keep around somehow more info around why the predicate failed and a timeout happened.
The idea of assertBusy is to allow to simply write "regular" test code, and if the test code trips, it will busy wait till a timeout. This allows us to keep around the assertion information and properly throw it for information that is inherently kept in the failure itself.
If you are indexing tiny documents then the previous default (5000)
was too low, causing excessive fsyncs with high indexing rates. With
this change we now only flush by byte size (200 MB) by default for
better indexing performance for tiny documents.
Closes#6783
When refresh_interval is long or disabled, and indexing rate is high,
it's possible for live version map to use non-trivial amounts of RAM.
With this change we now trigger a refresh in such cases to clear the
version map so we don't use unbounded RAM.
Closes#6443
Aggregation name are now able to use any character except '[', ']', and '>". Dot syntax may still be used to reference values (e.g. in the order field) but may only defence the value directly beneath the last aggregation in the list. more complex structures will need to be accessed using the aggname[value] syntax
Closes#6702
When a master receives a cluster state from another node, it compares the local cluster state with the one it got. If the local one has a higher version, it sends a JoinClusterRequest to the other master to tell it step down. Because our network layer is asymmetric, we need to make sure we're connected before sending.
Closes#6779
Concrete indices is now called multiple times when needed instead of changing what's inside the incoming request with the concrete indices. Ideally we want to keep the original aliases or indices or wildcard expressions in the request.
Also made sure that the check blocks is done against the concrete indices, which wasn't the case for delete index, delete mapping, open index, close index, types exists and indices exists.
Closes#6694Closes#6777
In #6706 we change the master validation to start pining immediately after a new master as ellected or a node joined. The idea is to have a quicker response to failures. This does however create a problem if the new master has yet fully processed it's ellection and responds to the ping with a NoLongerMasterException. This causes the source node to remove the current master and ellect another, only to find out it's not a master either and so forth. We are moving this change to the feature/improve_zen branch, where the improvements we made will cause the situation to be handled properly.
This reverts commit ae16956e07.
This reverts:
"Test: Temporarily change delete/put_mapping to wait for green": commit e408f8f638c2dd97a3ec86c8a9ac940f43ab37a0.
"[TEST] wait for green to update mapping": commit b3641a2ee6eb23318d49f5f04b39149e70c2b65b.
The change added in #6762 helps making sure the pending mapping updates are processed on all nodes to prevent moving shards to nodes which are not yet fully aware of the new mapping. However it introduced a racing condition delete_mapping operations, potentially causing a type to be added after it's deletion. This commit solves this by only sending a mapping update if the mapping source has actually changed.
Closes#6772
the default mapping is not merged, but updated in place, and only put mapping API can change it, no need to make sure it has been properly updated on the master. This can cause conflicts when a put mapping for it happens at the same time.
closes#6771
When we know the ID for the document we are about to index was
auto-generated, we don't need to acquire/release the per-ID lock,
which might provide small speedups during highly concurrent indexing.
Closes#6584
In order to have access to all codecs and handlers by netty, they
need to be exposed during shading, otherwise only the classes, which
are used by the built are exposed.
The match_phrase_prefix provided the same explanation as the match_phrase
query. There was no indication that the last term was run as a prefix
query.
This change marks the last term (or terms if there are multiple terms
in the same position) with a *
Closes#2449
During phase1 we copy over all lucene segments. These may refer to mapping updates that are still queued up to be sent to master. We must make sure those pending updates are processed before completing the relocation.
Relates to #6648Closes#6762
we need to make sure we wait for all threads to finish executing, since there might still be a thread around even post await (i.e. just starting) performing updates
the pending tasks api will now include the current executing tasks (with a proper marker boolean flag)
this will also help in tests that wait for no pending tasks, to also wait till the current executing task is done
closes#6744
When we have many new fields keep being introduced, the immutable open map we used becomes more and more expensive because of its clone characteristics, and we use it in several places.
The usage semantics of it allows us to also use a CHM if we want to, but it would be nice to still maintain the concurrency aspects of volatile immutable map when the number of fields is sane.
Introduce a new map like data structure, that can switch internally to CHM when a certain threshold is met.
Also add a benchmark class to exploit the many new field mappings use case, which shows significant gains by using this change, to a level where mapping introduction is no longer a significant bottleneck.
closes#6707
This method tires to optimize boolean queries if there is only
one clause. Yet BooleanQuery already does that internally This
optimization is unneeded.
Closes#6743
If you try to remove a plugin in read only dir, you get a successful result:
```
$ bin/plugin --remove marvel
-> Removing marvel
Removed marvel
```
But actually the plugin has not been removed.
When installing, if fails properly:
```
$ bin/plugin -i elasticsearch/marvel/latest
-> Installing elasticsearch/marvel/latest...
Failed to install elasticsearch/marvel/latest, reason: plugin directory /usr/local/elasticsearch/plugins is read only
```
This change throw an exception when we don't succeed removing the plugin.
Closes#6546.
Simon Willnauer
Shay Banon
Spencer Alger
mikemccand
Clinton Gormley
Colin Goodheart-Smithe
Boaz Leskes
javanna
Naoki Orii
Nick Steele
Alexander Reelsen
Lee Hinman
Andrii Gakhov
Carsten Brandt
Britta Weber
Shinsuke Sugaya
David Pilato
Martijn van Groningen