especially the tests that check for update of mapping, we need to make sure that the cluster is green so mappings won't get override, also, put mapping during index creation when possible
- adds support for multiple logging configurations under the config dir (will pick up any logging.xxx in the config folder tree)
- plugins can now define a top level config directory that will be copied under es config dir and will be renamed after the plugin name (same as the support we have the plugin "bin" dirs)
Closes#6802
There is a special type of request that tries to not allocate another buffer when sending bytes request (used by the public cluster state action). With the new pages bytes reference support, the content can already be a composite channel buffer, take that into account when building the actual composite buffer that will be sent over the network
closes#6756
Today, if we miss on a get on setting, we try its camel case version. The assumption is that in our code, we never use camel case to lookup a setting, but we want to support camel case if the user provided one.
This can be expensive (#toCamelCase) when the get on the setting is done in a tight call, which is evident when running the allocation deciders as part of the reroute logic.
Instead of doing the camel case check on get, prepare an additional map that includes all the settings that are provided as came case, and try and lookup from it if needed.
closes#6765
At the moment one can iterate the MapperService to go through all document mappers. This includes the document mapper of DEFAULT_MAPPING, which may be surprising and lead to unintended results. This commit removes the Iterable implementation and add a docMappers method that asks the caller to make an explicit choice
Closes#6793
We check if the version map needs to be refreshed after we released
the readlock which can cause the the engine being closed before we
read the value from the volatile `indexWriter` field which can cause an
NPE on the indexing thread. This commit also fixes a potential uncaught
exception if the refresh failed due to the engine being already closed.
Relates to #6443Closes#6786
remove internal callas on FieldMappers#Names, we properly reuse FieldMapper, so there is no need to try and call intern in order to reuse the names. This can be heavy with many fields and continuous mapping parsing.
closes#6747
We use awaitBusy in our tests, the problem is that we have to check if it awaited or not, and then try and keep around somehow more info around why the predicate failed and a timeout happened.
The idea of assertBusy is to allow to simply write "regular" test code, and if the test code trips, it will busy wait till a timeout. This allows us to keep around the assertion information and properly throw it for information that is inherently kept in the failure itself.
If you are indexing tiny documents then the previous default (5000)
was too low, causing excessive fsyncs with high indexing rates. With
this change we now only flush by byte size (200 MB) by default for
better indexing performance for tiny documents.
Closes#6783
When refresh_interval is long or disabled, and indexing rate is high,
it's possible for live version map to use non-trivial amounts of RAM.
With this change we now trigger a refresh in such cases to clear the
version map so we don't use unbounded RAM.
Closes#6443
Aggregation name are now able to use any character except '[', ']', and '>". Dot syntax may still be used to reference values (e.g. in the order field) but may only defence the value directly beneath the last aggregation in the list. more complex structures will need to be accessed using the aggname[value] syntax
Closes#6702
When a master receives a cluster state from another node, it compares the local cluster state with the one it got. If the local one has a higher version, it sends a JoinClusterRequest to the other master to tell it step down. Because our network layer is asymmetric, we need to make sure we're connected before sending.
Closes#6779
Concrete indices is now called multiple times when needed instead of changing what's inside the incoming request with the concrete indices. Ideally we want to keep the original aliases or indices or wildcard expressions in the request.
Also made sure that the check blocks is done against the concrete indices, which wasn't the case for delete index, delete mapping, open index, close index, types exists and indices exists.
Closes#6694Closes#6777
In #6706 we change the master validation to start pining immediately after a new master as ellected or a node joined. The idea is to have a quicker response to failures. This does however create a problem if the new master has yet fully processed it's ellection and responds to the ping with a NoLongerMasterException. This causes the source node to remove the current master and ellect another, only to find out it's not a master either and so forth. We are moving this change to the feature/improve_zen branch, where the improvements we made will cause the situation to be handled properly.
This reverts commit ae16956e07.
This reverts:
"Test: Temporarily change delete/put_mapping to wait for green": commit e408f8f638c2dd97a3ec86c8a9ac940f43ab37a0.
"[TEST] wait for green to update mapping": commit b3641a2ee6eb23318d49f5f04b39149e70c2b65b.
The change added in #6762 helps making sure the pending mapping updates are processed on all nodes to prevent moving shards to nodes which are not yet fully aware of the new mapping. However it introduced a racing condition delete_mapping operations, potentially causing a type to be added after it's deletion. This commit solves this by only sending a mapping update if the mapping source has actually changed.
Closes#6772
the default mapping is not merged, but updated in place, and only put mapping API can change it, no need to make sure it has been properly updated on the master. This can cause conflicts when a put mapping for it happens at the same time.
closes#6771
When we know the ID for the document we are about to index was
auto-generated, we don't need to acquire/release the per-ID lock,
which might provide small speedups during highly concurrent indexing.
Closes#6584
The match_phrase_prefix provided the same explanation as the match_phrase
query. There was no indication that the last term was run as a prefix
query.
This change marks the last term (or terms if there are multiple terms
in the same position) with a *
Closes#2449
During phase1 we copy over all lucene segments. These may refer to mapping updates that are still queued up to be sent to master. We must make sure those pending updates are processed before completing the relocation.
Relates to #6648Closes#6762
we need to make sure we wait for all threads to finish executing, since there might still be a thread around even post await (i.e. just starting) performing updates
the pending tasks api will now include the current executing tasks (with a proper marker boolean flag)
this will also help in tests that wait for no pending tasks, to also wait till the current executing task is done
closes#6744
When we have many new fields keep being introduced, the immutable open map we used becomes more and more expensive because of its clone characteristics, and we use it in several places.
The usage semantics of it allows us to also use a CHM if we want to, but it would be nice to still maintain the concurrency aspects of volatile immutable map when the number of fields is sane.
Introduce a new map like data structure, that can switch internally to CHM when a certain threshold is met.
Also add a benchmark class to exploit the many new field mappings use case, which shows significant gains by using this change, to a level where mapping introduction is no longer a significant bottleneck.
closes#6707
This method tires to optimize boolean queries if there is only
one clause. Yet BooleanQuery already does that internally This
optimization is unneeded.
Closes#6743
If you try to remove a plugin in read only dir, you get a successful result:
```
$ bin/plugin --remove marvel
-> Removing marvel
Removed marvel
```
But actually the plugin has not been removed.
When installing, if fails properly:
```
$ bin/plugin -i elasticsearch/marvel/latest
-> Installing elasticsearch/marvel/latest...
Failed to install elasticsearch/marvel/latest, reason: plugin directory /usr/local/elasticsearch/plugins is read only
```
This change throw an exception when we don't succeed removing the plugin.
Closes#6546.
Adding code to test for unset plugin names to fail fast with descriptive error messages. Also simplified the series of `if` statements checking for the commands by using a `switch` (now that it's using Java 7), added tests, and updated random exceptions with the up-to-date flag names (e.g., "--verbose" instead of "-verbose").
Closes#5976.
Closes#6013.
This causes a NPE since XContentStructure checks if the query is null
and takes this as the condition to parse from the byte source which is
actually null in that case.
Closes#6722
In this test we only index a handful of docs so if we have more shards
than docs we might fail on the `assertSearchResult` since not all shards
are started but results are just fine.
Today, we take great care to try and share the same analyzer instances across shards and indices (global analyzer). The idea is to share the same analyzer so the thread local resource it has will not be allocated per analyzer instance per thread.
The problem is that AnalyzerWrapper keeps its resources on its own per thread storage, and with per field reuse strategy, it causes for per field per thread token stream components to be used. This is very evident with the StandardTokenizer that uses a buffer...
This came out of test with "many fields", where the majority of 1GB heap was consumed by StandardTokenizer instances...
closes#6714
After a node joins the clusters, it starts pinging the master to verify it's health. Before, the cluster join request was processed async and we had to give some time to complete. With #6480 we changed this to wait for the join process to complete on the master. We can therefore start pinging immediately for fast detection of failures. Similar change can be made to the Node fault detection from the master side.
Closes#6706
The change introduced in #6686 checks for ConnectionTransportException during pinging. However, transport layer wraps it in SendRequestTransportException
Add `irish` analyzer
Add `sorani` analyzer (Kurdish)
Add `classic` tokenizer: specific to english text and tries to recognize hostnames, companies, acronyms, etc.
Add `thai` tokenizer: segments thai text into words.
Add `classic` tokenfilter: cleans up acronyms and possessives from classic tokenizer
Add `apostrophe` tokenfilter: removes text after apostrophe and the apostrophe itself
Add `german_normalization` tokenfilter: umlaut/sharp S normalization
Add `hindi_normalization` tokenfilter: accounts for hindi spelling differences
Add `indic_normalization` tokenfilter: accounts for different unicode representations in Indian languages
Add `sorani_normalization` tokenfilter: normalizes kurdish text
Add `scandinavian_normalization` tokenfilter: normalizes Norwegian, Danish, Swedish text
Add `scandinavian_folding` tokenfilter: much more aggressive form of `scandinavian_normalization`
Add additional languages to stemmer tokenfilter: `galician`, `minimal_galician`, `irish`, `sorani`, `light_nynorsk`, `minimal_nynorsk`
Add support access to default Thai stopword set "_thai_"
Fix some bugs and broken links in documentation.
Closes#5935
`-Dtests.filter` allows to pass filter expressions to the elasticsearch
tests. This allows to filter test annotaged with TestGroup annotations
like @Slow, @Nightly, @Backwards, @Integration with a boolean expresssion like:
* to run only backwards tests run:
`mvn -Dtests.bwc.version=X.Y.Z -Dtests.filter="@backwards"`
* to run all integration tests but skip slow tests run:
`mvn -Dtests.filter="@integration and not @slow"
* to take defaults into account ie run all test as well as backwards:
`mvn -Dtests.filter="default and @backwards"
This feature is a more powerful alternative to flags like
`-Dtests.nighly=true|false` etc.
Closes#6703
today we track both the index name and type for mapping updates in the shard bulk action, but we only work against on index in this level, so no need to track the index name itself
closes#6695
Both the Master and Node fault detection register themselves to be notified when a node disconnects to be able to respond to it accordingly. As such, when a ConnectionTransportException was raised on a ping request, it was not handled as it is already handled somewhere else. However, this does introduce a racing condition, if the disconnect happen during a period where there is no current master (minimum_master_node breach) at which time the fault detection is not active. In this case, we will only discover the disconnect error during the ping request, so we have to respond accordingly.
Closes#6686
randomization
We randomize the XContentType to test deriving the content type on all
APIs. Yet, BWC tests run against versions where CBOR wasn't around
this commit ensures we don't use CBOR when compatibility version is
less than `1.2.0`
Closes#6691
The concrete DocMapper on the master will be updated before the mapping in the cluster state. The DocMapper is updated during the cluster update task. This can lead to occasional assertion failures on the mapping response, because that is based on the mapping the cluster state, which may not yet have been updated. (time window between the DocMapping is updated, but the mapping in the cluster state isn't)
when an indexing request introduces a new mapping, today we rely on the parsing logic to mark it as modified on the "first" parsing phase. This can cause sending of mapping updates to master even when the mapping has been introduced in the create index/put mapping case, and can cause sending mapping updates without needing to.
This bubbled up in the disabled field data format test, where we explicitly define mappings to not have the update mapping behavior happening, yet it still happens because of the current logic, and because in our test we delay the introduction of any mapping updates randomly, it can get in and override updated ones.
closes#6669
In case an update request failed (for example when updating with a
wrongly formatted date), the returned index operation type was index
instead of update.
Closes#6630
At the moment the IndexingMemoryController can try to update the index buffer memory of shards at any give moment. This update involves a flush, which may cause a FlushNotAllowedEngineException to be thrown in a concurrently finalizing recovery.
Closes#6642, closes#6667
Due to a bogus if-check in SearchSourceBuilder.fetchSource(String include, String exclude)
the excludes only got set when the includes were not null. Fixed this and added some
basic tests.
Closes#6632
Sorting fails on unmapped fields so the new propagation delay of the mappings
exposed this issue. I added explicit mappings as part of index creation to fix it.
when the primary shard is recovering its translog, make sure to wait for new mapping introductions till the mappings have been updated on the master before finalizing the recovery itself
also, this change performs the mapping updates in a more optimized manner by batching the types to change into a single set and sending after the translog has been replayed
also, remove the wait for mapping on master in the local state tests since this new behavior covers it
closes#6666
remove waiting for mapping on master since we do it in recovery
Some IO api can return after writing & reading only a part of the requested data. On these rare occasions, we should call the methods again to read/write the rest of the data. This has cause rare translog corruption while writing huge documents on Windows.
Noteful parts of the commit:
- A new Channels class with utility methods for reading and writing to channels
- Writing or reading to channels is added to the forbidden API list
- Added locking to SimpleFsTranslogFile
- Removed FileChannelInputStream which was not used
Closes#6441 , #6576
All settings should be passes as settings and the enviroment should not
influence the test cluster settings. The settings we care about ie.
`es.node.mode` and `es.logger.level` should be passed via settings.
This allows tests to override these settings if they for instance need
`network` transport to operate at all.
Closes#6663
Today `index.version.created` depends on the version of the master
node in the cluster. This is potentially causing new features to be
expected on shards that didn't exist when the index was created.
There is no notion of `where was the shard allocated first` such that
`index.version.created` can't be reliably used as a feature flag.
With this change the `index.version.created` can be reliably used to
determin the smallest nodes version at the point in time when the index
was created. This means we can safely use certain features that would
for instance require reindeing and / or would not work if not the
entire index (all shards and segments) have been created with a certain
version or newer.
Closes#6660
When three threads are trying to write checksums at the same time, it's possible for all three threads to obtain the same checksum file name A. Then the first thread enters the synchronized section, creates the file with name A and exits. The second thread enters the synchronized section, checks that A exists, creates file A+1 and exits the critical section. Then it proceeds to clean up and deletes all checksum files including A. If it happens before the third thread enters the synchronized section, it's possible for the third thread to check for A and since it no longer exists create the checksum file A the second time, which triggers "file _checksums-XXXXXXXXXXXXX was already written to" exception in MockDirectoryWrapper and fails recovery.
They were due to a combination of mappings propagation delays and the behavior
of MapperService.smartName(String) so mappings are now configured up-front.
Makes it possible to delete snapshots that are missing some of the metadata files. This can happen if snapshot creation failed because repository drive ran out of disk space.
Closes#6383
The delayed mapping intro tests exposed a bug where if a new mapping is introduced, yet not updated on the master, and a full restart occurs, reply of the transaction log will not cause the new mapping to be re-introduced.
closes#6659
add comment on the method
Today, when a new mapping is introduced, the mapping is rebuilt (refreshSource) on the thread that performs the indexing request. This can become heavier and heavier if new mappings keeps on being introduced, we can move this process to another thread that will be responsible to refresh the source and then send the update mapping to the master (note, this doesn't change the semantics of new mapping introduction, since they are async anyhow).
When doing so, the thread can also try and batch as much updates as possible, this is handy especially when multiple shards for the same index exists on the same node. An internal setting that can control the time to wait for batches is also added (defaults to 0).
Testing wise, a new support method on ElasticsearchIntegrationTest#waitForConcreteMappingsOnAll to allow to wait for the concrete manifestation of mappings on all relevant nodes is added. Some tests mistakenly rely on the fact that there are no more pending tasks to mean mappings have been updated, so if we see, timing related, failures down later (all tests pass), then those will need to be fixed to wither awaitBusy on the master for the new mapping, or in the rare case, wait for the concrete mapping on all the nodes using the new method.
closes#6648
allow to change the additional time window dynamically
better sorting on mappers when refreshing source
also, no need to call nodes info in test, we already have the node names
clean calls to mapping update to provide doc mapper and UUID always
also use the internal cluster support method to get the list of nodes an index is on
reverse the order to pick the latest change first
remove unused field
and fix constructor param
move to start/stop on mapping update action
randomize INDICES_MAPPING_ADDITIONAL_MAPPING_CHANGE_TIME
Try and push our system to a state where there is only a single worker, trying to expose potential deadlocks when we by mistake execute blocking operations on the worker thread
closes#6635
only change recovery throttling to slow down recoveries. The recovery file chunk size updates are not picked up by ongoing recoveries. That cause the recovery to take too long even after the default settings are restored.
Also - change document creation to reuse field names in order to speed up the test.
We clone RateLimitedIndexOutput from lucene just to collect pausing
statistics we can do this in a more straight forward way in a delegating
RateLimiter.
Closes#6625
Thread rejection should return too many requests status code, and not 503, which is used to also show that the cluster is not available
relates to #6627, but only for rejections for now
closes#6629
We want to make sure recycling will not fail for any reason while trying to send a response back that is caused by a failure, for example, if we have circuit breaker on it (at one point), sending an error back will not be affected by it.
closes#6631
the test failed but couldn't repro (yet), at the very least, make sure we have the exception message as the reason, can help to track down the failure itself when it happens again
If the match query with cutoff_frequency encounters stacked tokens,
like synonyms in the same position, it returns a boolean query instead
of a common terms query. However, if the original operator was set
to "and", it was ignoring that and resetting the operator to "or".
In fact, if operator is "and" then there is little benefit in using
a common terms query as a must query is already
executed efficiently.
Waiting for ongoing recoveries was not good enough as it can run before the master finishing processing the started events of primary shards, causing the recovery response to be erroneously empty
Made AckedClusterStateUpdateTask an abstract class instead of an interface, which contains the common methods.
Also introduced the AckedRequest interface to mark both AcknowledgedRequest & ClusterStateUpdateRequest so that the different ways of updating the cluster state (with or without a MetaData*Service) can share the same code.
Removed ClusterStateUpdateListener as we can just use its base class ActionListener instead.
Closes#6559
Sandboxes the groovy scripting language with multiple configurable
whitelists:
`script.groovy.sandbox.receiver_whitelist`: comma-separated list of string
classes for objects that may have methods invoked.
`script.groovy.sandbox.package_whitelist`: comma-separated list of
packages under which new objects may be constructed.
`script.groovy.sandbox.class_whitelist` comma-separated list of classes
that are allowed to be constructed.
As well as a method blacklist:
`script.groovy.sandbox.method_blacklist`: comma-separated list of
methods that are never allowed to be invoked, regardless of target
object.
The sandbox can be entirely disabled by setting:
`script.groovy.sandbox.enabled: false`
Shay Banon
uboness
Areek Zillur
Valdis Rigdon
Clinton Gormley
Simon Willnauer
Martijn van Groningen
Boaz Leskes
Areek Zillur
mikemccand
Colin Goodheart-Smithe
javanna
Lee Hinman
Britta Weber
David Pilato
Chris Earle
Brusic
Robert Muir
Alexander Reelsen
Adrien Grand
Igor Motov
Andrew Raines