Improve IndexingMemoryController a bit:
- promptly push indexing buffer changes to IndexWriter, instead of waiting for next refresh/flush
- don't wait for merges to finish before dropping a shards's indexing buffer to 512 KB once it's inactive
- fix NPE if indices.memory.index_buffer_size is in node's settings with a bytes (not %) unit
- add some more logger.debug
This commit removes and now forbids all uses of
com.google.common.base.Joiner across the codebase. This is one of many
steps in the eventual removal of Guava as a dependency.
Relates #13224
This commit removes and now forbids all uses of
com.google.common.math.LongMath across the codebase. This is one step
of many in the eventual removal of Guava as a dependency.
This commit removes and now forbids all uses of
com.google.common.collect.Iterables across the codebase. This is one of
many steps in the eventual removal of Guava as a dependency.
Relates #13224
This commit removes and now forbids all uses of
com.google.common.base.Preconditions across the codebase. This is one
of many steps in the eventual removal of Guava as a dependency.
Relates #13224
When running a RestIT test from the IDE, you actually start an internal node which does not automatically load the plugin you would like to test.
We need to add:
```java
@Override
protected Collection<Class<? extends Plugin>> nodePlugins() {
return pluginList(PLUGIN_HERE.class);
}
```
Everything works fine when running from maven because each test basically:
* installs elasticsearch
* installs one plugin
* starts elasticsearch with this plugin loaded
* runs the test
Note that this PR only fixes the fact we run an internal cluster with the expected plugin.
Cloud tests will still fail when run from the IDE because is such a case you actually start an internal node with many mock plugins.
And REST test suite for cloud plugins basically checks if the plugin is running by checking the output of NodesInfo API.
And we check:
```yml
- match: { nodes.$master.plugins.0.name: cloud-azure }
- match: { nodes.$master.plugins.0.jvm: true }
```
But in that case, this condition is certainly false as we started also `mock-transport-service`, `mock-index-store`, `mock-engine-factory`, `node-mocks`, `asserting-local-transport`, `mock-search-service`.
Closes#13479
https://github.com/elastic/elasticsearch/pull/12908
Moved output to verbose level and additionally changed the plugin info output format
Before:
```shell
PluginInfo{name='cloud-aws', description='The Amazon Web Service (AWS) Cloud plugin allows to use AWS API for the unicast discovery mechanism and add S3 repositories.', site=false, jvm=true, classname=org.elasticsearch.plugin.cloud.aws.CloudAwsPlugin, isolated=true, version='2.1.0-SNAPSHOT'}
```
After:
```shell
- Plugin information:
Name: cloud-aws
Description: The Amazon Web Service (AWS) Cloud plugin allows to use AWS API for the unicast discovery mechanism and add S3 repositories.
Site: false
Version: 2.1.0-SNAPSHOT
JVM: true
* Classname: org.elasticsearch.plugin.cloud.aws.CloudAwsPlugin
* Isolated: true
```
Fixes#12907
If we don't wait we might retrieve the cluster state before second node
was added when we try to add the delegate in which the discovery node for
node_2 is null.
When a single node starts up it will first elect itself as master and then tries to
recover the cluster state or, if there is none, initialize an empty one and publish it.
Until it has done that, the cluster state will contain a global block and
requests might fail with
SERVICE_UNAVAILABLE/1/state not recovered / initialized
We need to wait for green.
* Automatic package repository creation for debian and rpm repositories using deb-s3 and rpm-s3 tools
* Fixing paths in email for repositories
* Add manual verification step for maven staging repo
* Do not create release directory in /tmp, because we might loose it on VMs
* Removed unused download-s3 script
* Add signage check for RPM
* Removed download-s3.py/upload-s3.py, as they are unused
Closes#13209
This commit fixes a test bug in
o.e.a.s.b.n.TransportBroadcastByNodeActionTests. Namely, the randomized
test allowed for the creation of cluster states that allocated indices
having zero shards. This ultimately surfaced in a
NoSuchElementException when attempting to iterate over the nonexistent
shards. The fix is merely to draw the random number of shards from 1 to
10 instead of 0 to 10.
This commit replaces the usage of LoadedCache with a simple CHM and calls
to computeIfAbsent and adds LoadingCache and CacheLoader to forbidden APIs
Relates to #13224
* Added BWC indices
* Added snapshot version to Version.java
* Fixed create_bwc_index to use localhost instead of localhost and 127.0.0.1 (problem with ipv4/6 setup)
This commit removes and now forbids all uses of
Function, Charsets, Collections2 across the codebase. This
is one of many steps in the eventual removal of Guava as a dependency.
Relates #13224
When publishing a new cluster state, the master will send it to all the node of the cluster, noting down how many *master* nodes responded successfully. The nodes do not yet process the new cluster state, but rather park it in memory. As soon as at least minimum master nodes have ack-ed the cluster state change, it is committed and a commit request is sent to all the node that responded so far (and will respond in the future). Once receiving the commit requests the nodes continue to process the cluster state change as they did before this change.
A few notable comments:
1. For this change to have effect, min master nodes must be configured.
2. All basic cluster state validation is done in the first phase of publish and is thus now part of `ShardOperationResult`
3. A new `COMMIT_TIMEOUT` settings is introduced, dictating how long a master should wait for nodes to ack the first phase. Unlike `PUBLISH_TIMEOUT`, if waiting for a commit times out, the cluster state change will be rejected.
4. Failing to achieve a min master node of acks, will cause the master to step down as it clearly doesn't have enough active followers.
5. Previously there was a short window between the moment a master lost it's followers and it stepping down because of node fault detection failures. In this short window, the master could process any change (but fail to publish it). This PR closes this gap to 0.
6. A dedicated pending cluster states queue was added to keep pending non-comitted cluster states and manage the logic around processing committed cluster states. See #13303 for details.
Closes#13062 , Closes#13303
This commit allows to refresh the info service in a blocking fashion
which allows tests to prevent installing listeners alltogether and
makes the class easier to test. Installing a listnener is always subject
to concurrent modifications where the listener might be called mulitple times
or with stale information which causes tests to fail.
Java 8 allows for method references which in-turn will cause
compile errors if a method is not visible while reflection fails late
and maybe too late. We can now register Request instances via FooRequest::new
instead of passing FooRequest.class and call it's ctor via reflection.
Jason Tedor
Michael McCandless
Pedro Andujar
Simon Willnauer
Christoph Büscher
David Pilato
Lee Hinman
Michael McCandless
Nik Everett
Britta Weber
Britta Weber
Alexander Reelsen
Robert Muir
Boaz Leskes
Simon Willnauer