The bulk processor tries to acquire all leases for the semaphore to wait
for all pending requests. Yet, we should release them afterwards again to
ensure we don't ever deadlock if there is a bug in the processor.
This commit also adds a testcase for this method
It's now possible to register watchers along with a specified check frequency. There are three frequencies: low, medium, high. Each one is associated with a check interval that determines how frequent the watchers will check for changes and notify listeners if needed. By default, the intervals are 5s, 30s and 60s respectively, but they can also be customized in the settings. also:
- Added the WatcherHandle construct by which one can stop it (remove it) and resume it (re add it). Also provices access to the watchers itself and the frequency by which it's checked
- Change the default frequency to 30 seconds interval (used to be 60 seconds). The only watcher that is currently effected by this is the script watcher (now auto-loading scripts will auto-load every 30 seconds if changed)
This is to prevent a rare racing condition where the very same shard gets allocated to the node after our sanity check that the cluster state didn't check and the actual deletion of the files.
Closes#6902
This results in unstable tests, most likely due to Channels being mixed
up by wrongly creating the pipelines. Needs investigation and a test.
This reverts commit db7f0d36af.
The IndexingMemoryController determines the amount of indexing buffer size and translog buffer size each shard should have. It takes memory from inactive shards (indexing wise) and assigns it to other shards. To do so it needs to know about the addition and closing of shards. The current implementation hooks into the indicesService.indicesLifecycle() mechanism to receive call backs, such shard entered the POST_RECOVERY state. Those call backs are typically run on the thread that actually made the change. A mutex was used to synchronize those callbacks with IndexingMemoryController's background thread, which updates the internal engines memory usage on a regular interval. This introduced a dependency between those threads and the locks of the internal engines hosted on the node. In a *very* rare situation (two tests runs locally) this can cause recovery time outs where two nodes are recovering replicas from each other.
This commit introduces a a lock free approach that updates the internal data structures during iterations in the background thread.
Closes#6892
FieldMapper has two methods
`Filter termsFilter(List values, @Nullable QueryParseContext)` which is supposed
to work on the inverted index and
`Filter termsFilter(QueryParseContext, List, QueryParseContext)` which is
supposed to work on field data. Let's rename the second one to
`fieldDataTermsFilter` and remove the unused `QueryParseContext`.
Close#6888
If you call `bin/plugin --remove es-plugin` the plugin got removed but the file `bin/plugin` itself was also deleted.
We now don't allow the following plugin names:
* elasticsearch
* plugin
* elasticsearch.bat
* plugin.bat
* elasticsearch.in.sh
* service.bat
Closes#6745
With the removal of setNextScore in #6864, script engines must use
the Scorer to find the score of a document. The DocLookup is updated
appropriately to do this, but most script engines require a Number to be
bound for numeric variables. Groovy already had an encapsulation for
this funtionality, and this moves it out to be shared with other script
engines.
closes#6898
While it would be nice to do this all the way up the chain (into
score functions), this at least removes the weird dual
setNextScore/setScorer for SearchScripts.
closes#6864
This fixes an issue introduced by the serialization changes in #6486
which are not needed at all. Node that the serialization itself is not broken
but the TransportClient uses its own version on initial connect and getting
the NodeInfos.
Today we synchronize when updating the IndexFieldDataService
datastructures. This might unnecessarily block progress if multiple
request need different fielddata instance for different fields.
This commit also fixes clear calls to actually consistently clear
the caches in the case of an exception.
Closes#6855
As a SizeValue is used for serializing the thread pool size, a negative number
resulted in throwing an exception when deserializing (using -ea an assertionerror
was thrown).
This fixes a check for changing the serialization logic, so that negative numbers are read correctly, by adding an internal UNBOUNDED value.
Closes#6325Closes#5357
Due to change introduced in #6825, we now start a local gateway recovery for replicas, if the source node can not be found. The recovery then fails because we never recover replicas from disk.
Closes#6879
In rare cases we may fail to send a shard failure event to the master, or there is no known master when the shard has failed (ex. a couple of node leave the cluster canceling recoveries and causing a master to step down at the same time). When that happens and a cluster state arrives from the (new) master we should resend the shard failure in order for the master to remove the shard from this node.
Closes#6881
make the builder releasable (auto closeable), and use it in shards state
also make XContentParser releasable (AutoCloseable) and not closeable since it doesn't throw an IOException
closes#6869
use similar mechanism that shares numeric analyzers for long/double/... for dates as well. This has nice memory save properties with many date fields mapping case, as well as analysis saves (thread local resources)
closes#6843
Enviorment variables might override the tests settings even if
they are explicitly set. Other base classes like InternalTestCluster
also specify `config.ignore_system_properties: true` to ensure `what
we set is what we get`
These are javascript expressions, which can only access numeric
fielddata, parameters, and _score. They can only be used for searches (not document updates).
closes#6818
This test makes it easy to create a lightweight node (no http, indices stored
in RAM, ...) whose main purpose is to get an instance of the Guice injector
for unit tests.
This should help not have to update lots of unit tests when we add a new
Guice dependency.
Throwables thrown on update retries are now caught and handled via
the provided callback. This commit also contains an integration test
demonstrating the bug and validating the fix.
Closes#6355Closes#6724
Each transport action is associated with at least an action name, which is the action name that gets serialized together with the request and identifies what to do with the request itself. Also, the action name is the name of the registered transport handler that handles incoming request for the transport action.
This commit makes the action name available in a generic manner in the TransportAction base class, so that it can be used when needed by subclasses, or in the base class for instance for action filtering.
Closes#6860
Today we close/reopen IW when we change the RAM buffer but that's
costly because it means the next NRT reader is a full reopen. The RAM
buffer size setting is a live one in IndexWriter, even if there are no
buffered docs in RAM when you call it.
Separately it would be nice if Lucene let you manage a "reader pool"
that could outlive individual IW instances ...
Closes#6856
Simon Willnauer
matt-preston
uboness
Boaz Leskes
Alexander Reelsen
Adrien Grand
Colin Goodheart-Smithe
Martijn van Groningen
David Pilato
Ryan Ernst
Bill Hwang
Clinton Gormley
Brian Murphy
Shay Banon
Nik Everett
mikemccand
Patrick Peschlow
javanna