We used to check on several places if we are still open but non of these
places did the check under the lock which leaves a small window where we
potentially get closed but still access an already closed channel or another
IO resource.
This is a pretty trivial change that moves most of the monitor service related
object creation from guice into the monitor service. This is a babystep towards removing
guice on the node level as well. Instead of opening huge PRs I try to do this in baby-steps
that are easier to digest.
We handle AlreadyClosedExceptions gracefully wherever IndexShard / Engine
is used. In some cases, instead of throwing the appropriate exception we
bubble up ChannelClosedException instead which causes shard failures etc.
Today, it seems like that this can only happen if the engine is closed without
acquireing the lock which means that the shard has failed already so the impact is really
just a confusing log message. Yet, this change enforces throwing the right exception
if the translog is already closed.
Closes#14866
This moves the registration of field mappers from the index level to the node
level and also ensures that mappers coming from plugins are treated no
differently from core mappers.
This commit fixes some leniency in the parsing of CIDRs. The leniency
that existed before includes not validating the octet values nor
validating the network mask; in some cases issues with these values were
silently ignored. Parsing is now done according to the guidelines in RFC
4632, page 6.
Closes#14862
For example: if a node left the cluster and an async store fetch was triggered. In that time no shard is marked as delayed (and strictly speaking it's not yet delayed). This caused test for shard delays post node left to fail. see : http://build-us-00.elastic.co/job/es_core_master_windows-2012-r2/2074/testReport/
To fix this, the delay update is now done by the Allocation Service, based of a fixed time stamp that is determined at the beginning of the reroute.
Also, this commit fixes a bug where unassigned info instances were reused across shard routings, causing calculated delays to be leaked.
Closes#14890
This switches query parsing from manual field parsing to using ParseField.
Also adds unit tests for each query that check original json can be parsed
into query builders.
Relates to #8964
We recently refactored the queries to make them parsable on the
coordinating note and adding serialization and equals/hashCode
capability to them. So far ShapeBuilders nested inside queries
were still transported as a byte array that needs to be parsed
later on the shard receiving the query. To be able to also
serialize geo shapes this way, we also need to make all the
implementations of ShapeBuilder implement Writable.
This PR adds this to PointBuilder and also adds tests for
serialization, equality and hashCode.
The work for #10708 requires tighter integration with the current shard routing of a shard. As such, we need to make sure it is set before the IndexService exposes the shard to external operations.
Closes#14918
Index constraints should remove indices in the response if the field to evaluate if empty. Index constraints can't work with that and it is the same as if the field doesn't match.
Currently we use the "gradle project attachment plugin" to support
building elasticsearch as part of another project. However, this plugin
has a number of issues, a large part of which is requiring consistent
use of the projectsPrefix.
This change removes projectsPrefix, and adds support for a special
extra-plugins directory in the root of elasticsearch. Any projects
checked out within this directory will be automatically added to
elasticsearch.
* Forbid System.setProperties & co in forbidden APIs.
* Ban property write access at runtime with security manager.
Plugins that need to modify system properties will need to request permission in their plugin-security.policy
This commit adds an acquired flag to BulkProcessor#execute that is set
only after successful acquisition of a permit on the semaphore
there. This flag is used to ensure that we do not release a permit on
the semaphore when we did not obtain a permit on the semaphore.
Closes#14908
This makes AvgTests use a mock plugin engine. I also removed the
textScriptExplicit* methods for the base class since they only make sense for
a groovy script, not a mock script.
After the removal of some internal shape builders in #14482 the
BaseLineStringBuilder has only one implementation, the LineStringBuilder.
Same for the BasePolygonBuilder. This PR removes the abstract classes
and merges them with their concrete implementation to simplify the
inheritance hierarchy.
The Ring subclass is just a LineStringBuilder that has an additional
close() method and keeps a reference to a parent shape builder so
builders can be chained. This PR removes it and replaces it by
using LineStringBuilder instead. The close() method is moved there
and tests are adapted.
This is a first step in reducing the number of ShapeBuilders since
before we start making the remaining implement Writable for the
search request refactoring. This shape builder seems to have been
only used in tests, and those tests didn't do much to begin with,
so this removed them.
Relates to #14416
- moves calculation of the delay to a single place (ReplicaShardAllocator)
- reduces coupling between GatewayAllocator and RoutingService
- in master failover situations, elapsed delay time is forgotten
Closes#14808
At the time of geo_shape query conception, CONTAINS was not yet a supported spatial operation in Lucene. Since it is now available this commit adds ShapeRelation.CONTAINS to GeoShapeQuery. Randomized testing is included and documentation is updated.
This commit sets ReplicationRequest.internalShardId to null when the
stream indicates that no ShardId is present in the stream.
Additionally, the use of StreamOutput#writeOptionalStreamable is
changed to be explicit for clarity since the use of
StreamInput#readOptionalStreamable is not possible due to the
no-argument constructor on ShardId being private.
This commit replaces all occurrences of Thread.interrupted() with
Thread.currentThread().interrupt(). While the former checks and clears the current
thread's interrupt flag the latter sets it, which is actually intended.
Closes#14798
This commit adds a unit test for LinkedHashMap serialization that tests
that the method of serialization writes the entries in the LinkedHashMap
in iteration order and that the reconstructed LinkedHashMap preserves
that order. This test is randomized and tests iteration order is
preserved whether the LinkedHashMap is ordered by insertion order or
access order.
Closes#14743
This commit adds a timeout mechanism for sending shard failures. The
requesting thread can attach a listener to the timeout event so that
handling it is part of the event chain.
Relates #14252
This commit changes the signature of StreamInput#readOptionalStreamable
to accept a Supplier to create new streamables rather than requiring
callers to construct new instances. This has the advantage of avoiding
an allocation in cases when the stream indicates the resulting
streamable is null
If you build elasticsearch without a git repository it was creating a null
shortHash which was causing Elasticsearch not to be able to form transport
connections.
Closes#14748
This commit adds a method of encoding longs using a variable-length
representation. This encoding is an implementation of the zig-zag
encoding from protocol buffers. Numbers that have a small absolute value
will use a small number of bytes. This is achieved by zig-zagging
through the space of longs in order of increasing absolute value (0, -1,
1, -2, 2, …, Long.MAX_VALUE, Long.MIN_VALUE) -> (0, 1, 2, 3, 4, …, -2,
-1). The resulting values are then encoded as if they represent unsigned
numbers.
ClusterStatsIT#testClusterStatus() contained a race where the
test cluster might still be initializing while test already checks
for a green health status.
With this commit the test waits until the cluster status changed and
checks health afterwards.
Checked with @bleskes.
This issue occurs if the center latitude of the GeoPointDistance query is set to one of the poles. Since this issue is set to be fixed in LUCENE-6897 this commit temporarily limits the random latitudinal location to not include the poles.
_type should have got doc values with the change to default doc values.
However, due to how metadata fields have separate builders and special
constructors, it was not picking it up. This change updates the field
type for _type to have doc values.
closes#14781
If we run out of disk while recoverying the transaction log
we repeatedly fail since we expect the latest tranlog to be uncommitted.
This change adds 2 safety levels:
* uncommitted checkpoints are first written to a temp file and then atomically
renamed into a committed (recovered) checkpoint
* if the latest uncommitted checkpoints generation is already recovered it has to be
identical, if not the recovery fails
This allows to fail in between recovering the latest uncommitted checkpoint and moving
the checkpoint generation to N+1 which can for instance happen in a situation where
we can run out of disk. If we run out of disk while recovering the uncommitted checkpoint
either the temp file writing or the atomic rename will fail such that we never have a
half written or corrupted recovered checkpoint.
Close#14695
Currently the abstract ShapeBuilder class serves too many different
purposes, making it hard to refactor and maintain the code. In order
to reduce the size and responsibilities, this PR moved all the
static factory methods used as a shortcut to create new shape builders
out to a new ShapeBuilders class, similar to how QueryBuilders is
used already.
Robert Muir
Jason Tedor
Simon Willnauer
Simon Willnauer
Britta Weber
Robert Muir
Christoph Büscher
Adrien Grand
Isabel Drost-Fromm
Clinton Gormley
Yannick Welsch
Michael McCandless
Daniel Mitterdorfer
Boaz Leskes
Isabel Drost-Fromm
Dominik Stadler
Martijn van Groningen
Ryan Ernst
Camilo Díaz Repka
markharwood
Colin Goodheart-Smithe
William Bowling
Xu Zhang
Nicholas Knize
Nik Everett
Michael McCandless
Tanguy Leroux