Folds the helper class for random object generation into the
abstract sort test class. Removes a few references to ESTestCase
that were not needed due to inheriting from it along the way.
The query shard reset() method resets some internal state in the
query shard context, like clearing query names, the filter flag
or named queries. The problem with this method being public is
that it currently (miss?) used for modifying an existing context
for recursive invocatiob, but the contexts that have been reseted
that way cannot be properly set back to their previous state.
This PR is a step towards removing reset() entirely by first making
it only be used internally in QueryShardContext. In places where
reset() was used we can either create new QueryShardContexts or
modify the existing context because it is discarded afterwards anyway.
Today, the constructor for IngestDocument#FieldPath does a string
concatentation and two object allocation on every field path. This
commit removes these unnecessary operations.
Relates #18108
With this commit we compress HTTP responses provided the client
supports it (as indicated by the HTTP header 'Accept-Encoding').
We're also able to process compressed HTTP requests if needed.
The default compression level is lowered from 6 to 3 as benchmarks
have indicated that this reduces query latency with a negligible
increase in network traffic.
Closes#7309
Don't try to compute completion stats on a reader after we already closed it
Conflicts:
core/src/main/java/org/elasticsearch/index/shard/IndexShard.java
This commit removes an unnecessary if statement in Bootstrap#check. The
removed if statement was duplicating the conditionals in the nested if
statements and was merely an artifact of an earlier refactoring.
Today when running in production mode the bootstrap checks are
completely unforgiving. But there are cases where an end-user might not
have the ability to modify some of the system-level settings that cause
the bootstrap checks to trip (e.g., guest settings that are inherited
from a host and can not be modified). This commit adds a setting that
allows system-level bootstrap checks to be ignored for these
end-users. We classify certain bootstrap checks into system-level checks
and only those bootstrap checks will be ignored if this flag is
enabled. All other bootstrap checks are still subject to being enforced
if the user is in production mode. We will still log warnings for these
bootstrap checks because the end-user does still need to be made aware
that they are running in a configuration that is less-than-ideal from a
resiliency perspective.
Relates #18088
This commit removes a racy but unnecessary assertion in scaling thread
pool idle test. Namely, the main test thread can reach the removed
assertion before the last few threads in the thread pool have completed
their tasks and caused the completed tasks count on the underlying
executor to be updated. But this assertion is unnecessary. The main test
thread already waits on a latch that is only decremented immediately
before a task completes. This ensures that it was in fact the case that
every submitted task was executed.
Closes#18072
When the termslookup (mocked in this case) doesn't return any terms, the
query used to rewrite to an empty boolean query. Now it rewrites to a
MatchNoDocsQuery. This changes the test expectation accordingly.
Closes#18071
This commit modifes the EsThreadPoolTestCase#info helper method to
return null when info for the thread pool can not be found. This really
should only happen for the "same" thread pool, and so we also assert
that we only get to a place where there is no info if the thread pool
that info was requested for is in fact the "same" thread pool. Not
returning null here and instead throwing an exception would fail tests
that tried to lookup info on the "same" thread pool.
Today we use a sliced lock strategy for acquiring locks to prevent
concurrent updates to the same document. The number of sliced locks is
computed as a linear function of the number of logical
processors. Unfortunately, the probability of a collision against a
sliced lock is prone to the birthday problem and grows faster than
expected. In fact, the mathematics works out such that for a fixed
target probability of collision, the number of lock slices should grow
like the square of the number of logical processors. This is
less-than-ideal, and we can do better anyway. This commit introduces a
strategy for avoiding lock contention within the internal
engine. Ideally, we would only have lock contention if there were
concurrent updates to the same document. We can get close to this ideal
world by associating a lock with the ID of each document. This
association can be held in a concurrent hash map. Now, the JDK
ConcurrentHashMap also uses a sliced lock internally, but it has several
strategies for avoiding taking the locks and these locks are only held
for a very short period of time. This implementation associates a
reference count with the lock that is associated with a document ID and
automatically removes the document ID from the concurrent hash map when
the reference count reaches zero.
Relates #18060
Fix a limitation that prevent from hierarchical inner hits be defined in query dsl.
Removed the nested_path, parent_child_type and query options from inner hits dsl. These options are only set by ES
upon parsing the has_child, has_parent and nested queries are using their respective query builders.
These options are still used internally, when these options are set a new private copy is created based on the
provided InnerHitBuilder and configuring either nested_path or parent_child_type and the inner query of the query builder
being used.
Closes#11118
Previously like in other geo related query parsers we were using
a combination of two booleans for coerce and ignore_malformed
which was error prone and not very clear.
Switched to using GeoValidationMethod instead as we already do
e.g. in GeoBoundingBoxQueryBuilder.
Left support for both, coerce and ignore_malformed in the parser
but deprecated the two in favour of validation method.
Introduced the same deprecation in geo bounding box query builder.
While returning no hits on fields that are not mapped may be fine, it is not
for fields that are mapped but not indexed (`index:false`). We should fail the
query in that case rather than returning no hits.
Switch something from an explicit toString to Strings.toString which
is the same thing but with more code reuse.
Also renamed a constant to be CONSTANT_CASE.
ObjectParser makes parsing XContent 95% easier. No more nested loops.
No more forgetting to use ParseField. Consistent handling for arrays.
Awesome. But ObjectParser doesn't support building things objects whose
constructor arguments are mixed in with the rest of its properties.
Enter ConstructingObjectParser! ConstructingObjectParser queues up
fields until all of the constructor arguments have been parsed and
then sets them on the target object.
Closes#17352
* Add isSearchable and isAggregatable (collapsed to true if any of the instances of that field are searchable or aggregatable).
* Accept wildcards in field names.
* Add a section named conflicts for fields with the same name but with incompatible types (instead of throwing an exception).
This commit fixes a test bug in the scaling thread pool idle
test. Namely, a random thread pool is chosen which could have a min pool
size of one or four but the while loop was acting as if the min pool
size was four (this is due to the test having been initially written for
only the generic thread pool).
Additionally, a latch is added between the test thread and the work
tasks to reduce the chance of a race condition between the test thread
and last few tasks.
This commit slightly expands the scaling thread pool configuration test
coverage. In particular, the test testScalingThreadPoolConfiguration is
expanded to include the case when min is equal to size, and the test
testDynamicThreadPoolSize is expanded to include all possible cases when
size is greater than or equal to min.
This commit fixes an index name equality check in RoutingNodes. Namely,
the check was comparing an instance of Index to an instance of
String. Instead, the index name should be obtained from the Index
instance to be compared to the instance of String.
Closes#17982
Previously, we would determine index deletes in the cluster state by
comparing the index metadatas between the current cluster state and the
previous cluster state and decipher which ones were missing (the missing
ones are deleted indices). This led to a situation where a node that
went offline and rejoined the cluster could potentially cause dangling
indices to be imported which should have been deleted, because when a node
rejoins, its previous cluster state does not contain reliable state.
This commit introduces the notion of index tombstones in the cluster
state, where we are explicit about which indices have been deleted.
In the case where the previous cluster state is not useful for index
metadata comparisons, a node now determines which indices are to be
deleted based on these tombstones in the cluster state. There is also
functionality to purge the tombstones after exceeding a certain amount.
Closes#17265Closes#16358Closes#17435
This adds information similar to what is from the [shard stores
API](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/indices-shards-stores.html)
to the cluster allocation explanation API (in fact, internally it uses
that API).
This means when you have a decision that otherwise could indicate that a
shard can go somewhere, you now have more information:
```json
{
"shard" : {
"index" : "i",
"index_uuid" : "QzoKda9aQCG_hCaZQ18GEg",
"id" : 0,
"primary" : true
},
"assigned" : false,
"unassigned_info" : {
"reason" : "CLUSTER_RECOVERED",
"at" : "2016-04-11T20:58:04.088Z"
},
"allocation_delay" : "0s",
"allocation_delay_ms" : 0,
"remaining_delay" : "0s",
"remaining_delay_ms" : 0,
"nodes" : {
"24Qmw4tdRTuVOtjAdtmr5Q" : {
"node_name" : "Vampire by Night",
"node_attributes" : { },
"final_decision" : "YES",
"weight" : 7.0,
"decisions" : [ ],
"store" : {
"allocation_id" : "aC6qVWA7TT2pgsalYxxUJQ",
"store_exception" : "IndexFormatTooOldException[Format version is not supported (resource BufferedChecksumIndexInput(SimpleFSIndexInput(path=\"/home/hinmanm/scratch/elasticsearch-5.0.0-alpha1-SNAPSHOT/data/elasticsearch/nodes/0/indices/QzoKda9aQCG_hCaZQ18GEg/0/index/segments_1\"))): -1906795950 (needs to be between 1071082519 and 1071082519). This version of Lucene only supports indexes created with release 5.0 and later.]",
"allocation" : "UNUSED"
}
}
}
}
```
The "store" section is the new section, and will include allocation, id,
and the exception if there is one.
Relates to #17372
This commit converts the settings for the ResourceWatcherService to use the new infrastructure and
registers the settings so that they do not cause errors when used.
This commit clarifies an error message that is produced when an attempt
is made to resize the backing queue for a scaling executor. As this
queue is unbounded, resizing the backing queue does not make sense. The
clarification here is to specify that this restriction is because the
executor is a scaling executor.
This commit actually bounds the size of the generic thread pool. The
generic thread pool was of type cached, a thread pool with an unbounded
number of workers and an unbounded work queue. With this commit, the
generic thread pool is now of type scaling. As such, the cached thread
pool type has been removed. By default, the generic thread pool is
constructed with a core pool size of four, a max pool size of 128 and
idle workers can be reaped after a keep-alive time of thirty seconds
expires. The work queue for this thread pool remains unbounded.
The list settings parser supports retrieving lists defined in settings that use a key followed by a `.` and a
number (for example `foo.bar.0`). However, the exists method would indicate that the provided settings
do not contain a value for this setting. This change makes it so that the exists method now handles this
format.
When a cli throws a USAGE error, it is implied that the user did
something wrong, and probably needs help in understanding the cli
arguments. This change adds help output before the usage error is
printed.
Do this by creating a Client subclass that automatically assigns the
parentTask to all requests that come through it. Code that doesn't want
to set the parentTask can call `unwrap` on the Client to get the inner
client instance that doesn't set the parentTask. Reindex uses this for
its ClearScrollRequest so that the request will run properly after the
reindex request has been canceled.
Now that the current uses of magical camelCase support have been
deprecated, we can remove these in master (sans remaining issues like
BulkRequest). This change removes camel case support from ParseField,
query types, analysis, and settings lookup.
see #8988
Callers should explicitly handle parents - either using EMPTY_TASK_ID when
a parent isn't possible or piping parents from the TransportRequest when
possible.
Isabel Drost-Fromm
Christoph Büscher
Jason Tedor
Yannick Welsch
Daniel Mitterdorfer
Mike McCandless
Michael McCandless
Adrien Grand
Ali Beyad
Martijn van Groningen
David Pilato
Tal Levy
Lee Hinman
Jay Modi
Mathias Fußenegger
Nik Everett
Jim Ferenczi
Clinton Gormley
Alexander Reelsen
Areek Zillur
jaymode
Ryan Ernst