This commit removes all special file handling from DistributorDirectory
that assigned certain files to the primary directory. This special handling
was added to ensure that files that are written more than once are essentially
overwritten. Yet this implementation is consistent all the time and doesn't need
this special handling for files that are written through this directory. Writes
to the underlying directory not going through the distributor directory are not
and have never been supported.
Note: this commit also fixes the problem of adding directories to the distributor
during restart where the primary can suddenly change and file mappings are by-passed.
Closes#8276
Also added:
* Better exception handling in UnicastZenPing#ping and MulticastZenPing#ping
* In the join thread that runs the innerJoinCluster loop, remember the last known exception and throw that when assertions are enabled. We loop until inner join has completed and if continues exceptions are thrown we should fail the test, because the exception shouldn't occur in production (at least not too often).
Applied feedback 3
Closes#8327
This commit adds the ability to associate a bit of state with each
individual aggregation.
The aggregation response can be hard to stitch back together without
having a reference to the aggregation request. In many cases this is not
available, many json serializer frameworks cache types globally or have a
static deserialisation override mechanism. In these cases making the
original request available, if at all possible, would be a hack.
The old facets returned `_type` which was just enough metadata to know
what the originating facet type in the request was.
This PR takes `_type` one step further by introducing ANY arbitrary meta
data. This could be further <strike>ab</strike>used for instance by
generic/automated aggregations that include UI state (color information,
thresholds, user input states, etc) per aggregation.
The discovery.zen.minimum_master_nodes setting can be updated dynamically. Settings it to a value higher then the current number of master nodes will cause the current master to step down. This is dangerous because if done by mistake (typo) there is no way to restore the settings (this requires an active master).
Closes#8321
version.
An early version of #7966 had the ability to choose a bwc version
automatically, but this was removed before the change was committed.
However, the change was not removed from the ongoing work in #7922
and it made it in unknowningly.
If includes or excludes are set
XContentFactory.xcontentBuilder() allocates a new
BytesStreamOutput using the default page size which is 16kb.
Can be optimized to use the length of the sourceRef because
that is the maximum possible size that the streamOutput will
use.
This redcues the amount of memory allocated for a request
that is fetching 200.000 small documents (~150 bytes each)
by about 300 MB
Close#8138
This adds HTTP pipelining support to netty. Previously pipelining was not
supported due to the asynchronous nature of elasticsearch. The first request
that was returned by Elasticsearch, was returned as first response,
regardless of the correct order.
The solution to this problem is to add a handler to the netty pipeline
that maintains an ordered list and thus orders the responses before
returning them to the client. This means, we will always have some state
on the server side and also requires some memory in order to keep the
responses there.
Pipelining is enabled by default, but can be configured by setting the
http.pipelining property to true|false. In addition the maximum size of
the event queue can be configured.
The initial netty handler is copied from this repo
https://github.com/typesafehub/netty-http-pipeliningCloses#2665
This changes the weighing function for the filter cache to use a
configurable minimum weight for each filter cached. This value defaults
to 1kb and can be configured with the
`indices.cache.filter.minimum_entry_weight` setting.
This also fixes an issue with the filter cache where the concurrency
level of the cache was exposed as a setting, but not used in cache
construction.
Relates to #8268
This adds a Listener interface to the ClusterInfoService, this is used
by the DiskThresholdDecider, which adds a listener to check for nodes
passing the high watermark. If a node is past the high watermark an
empty reroute is issued so shards can be reallocated if desired.
A reroute will only be issued once every
`cluster.routing.allocation.disk.reroute_interval`, which is "60s" by
default.
Refactors InternalClusterInfoService to delegate the nodes stats and
indices stats gathering into separate methods so they have be overriden
by extending classes. Each stat gathering method returns a
CountDownLatch that can be used to wait until processing for that part
is successful before calling the listeners.
Fixes#8146
As that environment variable can contain elasticsearch specific configuration
like the path to a configuration, it makes sense to support in both
elasticsearch commands.
Closes#8288
This change corrects the location information gathered by the loggers so that when printing class name, method name, and line numbers in the log pattern, the information from the class calling the logger is used rather than a location within the logger itself.
A reset method has also been added to the LogConfigurator class which allows the logging configuration to be reset. This is needed because if the LoggingConfigurationTests and Log4jESLoggerTests are run in the same JVM the second one to run needs to be able to override the log configuration set by the first
Closes#5130, #8052
we current check that the recovery is not finished when people access the status local variables. This is wrong and we should check for the refcount being > 0 as it is OK to use the status after it has marked as finished but there are still on going, in-flight reference to it.
Relates to #8092Closes#8271
This change corrects the location information gathered by the loggers so that when printing class name, method name, and line numbers in the log pattern, the information from the class calling the logger is used rather than a location within the logger itself.
Closes#5130
Boaz Leskes
Simon Willnauer
javanna
Adrien Grand
Martijn van Groningen
Martijn Laarman
Ryan Ernst
Robert Muir
Lee Hinman
printercu
Alexander Reelsen
Aarni Koskela
Alex Leonhardt
Mathias Fussenegger
Clinton Gormley
Oliver Eilhard
astefan
David Pilato
Satoyuki Tsukano
Colin Goodheart-Smithe
Michael McCandless
cmpich