With this commit we add a precondition check to BulkRequest so
we fail early if users pass `null` for the request object.
For a more detailed discussion, see #12038.
This supersedes #12038.
Relates #12038.
Today, a race condition exists when retiring executors. Namely, if an
executor is retired and then the thread pool is terminated, the retiring
of the executor and the termination of the thread pool can race to
remove the retired executor from the queue of retired executors. More
precisely, when the executor is initially retired, it is placed on a
queue of retired executors, and then removed when it is successfully
shutdown. When the pool is terminated, it will also drain the queue of
retired executors. This leads to a time-of-check-time-of-use race where
the draining can see a retired executor on the queue but that retired
executor can be removed upon successful shutdown of that executor. This
leads to the draining attempting to remove an element from the queue
when there is none. This commit addresses this race condition by instead
safely polling the queue.
Relates #18333
Previously multiple extensions could be provided, however, this can lead
to confusion with on-disk scripts (ie, "foo.js" and "foo.javascript")
having different content. Only a single extension is now supported.
The only language currently supporting multiple extensions was the
Javascript engine ("js" and "javascript"). It now only supports the
`.js` extension.
Relates to #10598
Currently `fuzziness` is not supported for the `cross_fields` type
of the `multi_match` query since it complicates the logic that
blends the term queries that cross_fields uses internally. At the
moment using this combination is silently ignored, which can lead to
confusions. Instead we should throw an exception in this case.
The same is true for phrase and phrase_prefix type.
Closes#7764
This commit adds a test that a fixed executors rejected count behaves as
expected. In particular, we test that if we consume the executor, then
stuff the executor queue, further tasks will be rejected and the
rejected stats are updated appropriately. This test also asserts that if
we resize the queue the rejected count is reset to zero.
Relates #18301
This removes all the mentions of the sandbox from the script engine
services and permissions model. This means that the following settings
are no longer supported:
```yaml
script.inline: sandbox
script.stored: sandbox
```
Instead, only a `true` or `false` value can be specified.
Since this would otherwise break the default-allow parameter for
languages like expressions, painless, and mustache, all script engines
have been updated to have individual settings, for instance:
```yaml
script.engine.groovy.inline: true
```
Would enable all inline scripts for groovy. (they can still be
overridden on a per-operation basis).
Expressions, Painless, and Mustache all default to `true` for inline,
file, and stored scripts to preserve the old scripting behavior.
Resolves#17114
Currently terms on an ip address try to put their binary representation in the
json response. With this commit, they would return a formatted ip address:
```
"buckets": [
{
"key": "192.168.1.7",
"doc_count": 1
}
]
```
We add support to explicitly exclude specific transport actions
from the request size limit check.
We also exclude the following request types currently:
*MasterPingRequest
* PingRequest
As most of our log messages are not sentences and do not end with
periods, this commit removes a period from the end of the min master
node bootstrap check log message.
Rearranges the FingerprintAnalyzer so that AsciiFolding comes earlier in the chain (after lowercasing, before stop removal, for maximum deduping power)
Closes#18266
This commit ensures that if CORS is enabled, then Origin headers are
checked regardless of whether the request came from a browser or not.
In the past, we only proceeded with CORS checks if the User-Agent was a
browser.
(same name and UUID) exists in the cluster state. This resolves a
situation where if an index data folder was copied into a node's data
directory while the node is running and that index had a tombstone in
the cluster state, the index would still get imported.
Closes#18250Closes#18249
It will keep using the caching terms enum for keyword/text fields and falls back
to IndexSearcher.count for fields that do not use the inverted index for
searching (such as numbers and ip addresses). Note that this probably means that
significant terms aggregations on these fields will be less efficient than they
used to be. It should be ok under a sampler aggregation though.
This moves tests back to the state they were in before numbers started using
points, and also adds a new test that significant terms aggs fail if a field is
not indexed.
In the long term, we might want to follow the approach that Robert initially
proposed that consists in collecting all documents from the background filter in
order to compute frequencies using doc values. This would also mean that
significant terms aggregations do not require fields to be indexed anymore.
This commit modifies two logging statements in the
IndexingMemoryController to log the key for the setting
indices.memory.index_buffer_size instead of the object.
Relates #18191
An additional sanity check introduced by #17821 makes some tests fail. This check verifies that
only one shard with same shard id is allocated to a node. This commit fixes a bug in
ClusterStateCreationUtils which would construct a cluster state that allocated two shards with same
id to the same node.
Today when join validation fails, we log a warning but do not log the
exception that led to the join validation failing. This commit modifies
this so that we do log this exception.
All implementations of SearchParseElement have been removed since they are no longer used now that parsing is done on the coordinating node. The SearchParseElement and FetchSubPhaseParseElement classes are not removed as currently they are needed for plugins that add a custom fetch sub phase. These will be removed in a follow up PR that will allow fetch sub phase plugins to register a parser in a different way.
The current code tries to handle the case that document versions are either
missing or stored in payloads rather than doc values. Since none of the 2.x
releases allowed this, we can remove this logic.
This removes dead/duplicate code and makes the `_index` field not configurable.
(Configuration used to jus be ignored, now we would throw an exception if any
is provided.)
With this commit we eagerly evaluate content length in HttpServer
and also pass the same value to ResourceHandlingHttpChannel. With
this change it easier to reason about the content length that is
freed leaving no doubt that it must be identical to the reserved
amount.
Most of the current implementations of BaseNodesResponse (plural Nodes) ignore FailedNodeExceptions.
- This adds a helper function to do the grouping to TransportNodesAction
- Requires a non-null array of FailedNodeExceptions within the BaseNodesResponse constructor
- Reads/writes the array to output
- Also adds StreamInput and StreamOutput methods for generically reading and writing arrays
This commit removes handshaking from the transport client. This
handshaking is not needed because of the existence of the liveness
check.
Relates #18174
Due to trying to modify a map while iterating it, a concurrent modification
in the pipeline stats could be thrown. This uses an iterator to prevent this.
Closes#18126
We should prevent highlighting if a field is anything but a text or keyword field.
However, someone might implement a custom field type that has text and still want to
highlight on that. We cannot know in advance if the highlighter will be able to
highlight such a field and so we do the following:
If the field is only highlighted because the field matches a wildcard we assume
it was a mistake and do not process it.
If the field was explicitly given we assume that whoever issued the query knew
what they were doing and try to highlight anyway.
closes#17537
The `ip` field uses a binary representation internally. This breaks when
rendering sort values in search responses since elasticsearch tries to write a
binary byte[] as an utf8 json string. This commit extends the `DocValueFormat`
API in order to give fields a chance to choose how to render values.
Closes#6077
QueryBuilder has generics, but those are never used: all call sites use
`QueryBuilder<?>`. Only `AbstractQueryBuilder` needs generics so that the base
class can contain a default implementation for setters that returns `this`.
In preparation for a unified release process, we need to be able to
generate the pom files independently of trying to actually publish. This
change adds back the maven-publish plugin just for that purpose. The
nexus plugin still exists for now, so that we do not break snapshots,
but that can be removed at a later time once snapshots are happenign
through the unified tools. Note I also changed the dir jars are written
into so that all our artifacts are under build/distributions.
This gives better coverage and consistency with the scripting APIs, by
whitelisting the primary search scripting API classes and using them instead
of only Map and List methods.
For example, accessing fields can now be done with `.value` instead of `.0`
because `getValue()` is whitelisted. For now, access to a document's fields in
this way (loads) are fast-pathed in the code, to avoid dynamic overhead.
Access to geo fields and geo distance functions is now supported.
TODO: date support (e.g. whitelist ReadableDateTime methods as a start)
TODO: improve docs (like expressions and groovy have for document's fields)
TODO: remove fast-path hack
Closes#18169
Squashed commit of the following:
commit ec9f24b2424891a7429bb4c0a03f9868cba0a213
Author: Robert Muir <rmuir@apache.org>
Date: Thu May 5 17:59:37 2016 -0400
cutover to <Def> instead of <Object> here
commit 9edb1550438acd209733bc36f0d2e0aecf190ecb
Author: Robert Muir <rmuir@apache.org>
Date: Thu May 5 17:03:02 2016 -0400
add fast-path for docvalues field loads
commit f8e38c0932fccc0cfa217516130ad61522e59fe5
Author: Robert Muir <rmuir@apache.org>
Date: Thu May 5 16:47:31 2016 -0400
Painless: add fielddata accessors (.value/.values/.distance()/etc)
o/e/snapshots/Snapshot and o/e/snapshots/SnapshotInfo contain the same
fields and represent the same information. Snapshot was used to
maintain snapshot information to the snapshot repository, while
SnapshotInfo was used to represent the snapshot information as presented
through the REST layer. This removes the Snapshot class and combines
all uses into the SnapshotInfo class.
Closes#18167
Adds infrastructure so `gradle :docs:check` will extract tests from
snippets in the documentation and execute the tests. This is included
in `gradle check` so it should happen on CI and during a normal build.
By default each `// AUTOSENSE` snippet creates a unique REST test. These
tests are executed in a random order and the cluster is wiped between
each one. If multiple snippets chain together into a test you can annotate
all snippets after the first with `// TEST[continued]` to have the
generated tests for both snippets joined.
Snippets marked as `// TESTRESPONSE` are checked against the response
of the last action.
See docs/README.asciidoc for lots more.
Closes#12583. That issue is about catching bugs in the docs during build.
This catches *some* bugs in the docs during build which is a good start.
Today we softly warn about running with the client VM. However, we
should really refuse to start in production mode if running with the
client VM as the performance of the client VM is too devastating for a
server application. This commit adds an option to jvm.options to ensure
that we are starting with the server VM (on all 32-bit non-Windows
platforms on server-class machines (2+ CPUs, 2+ GB physical RAM) this is
the default and on all 64-bit platforms this is the only option) and
adds a bootstrap check for the client VM.
Relates #18155
This commit introduces a handshake when initiating a light
connection. During this handshake, node information, cluster name, and
version are received from the target node of the connection. This
information can be used to immediately validate that the target node is
a member of the same cluster, and used to set the version on the
stream. This will allow us to extend APIs that are used during initial
cluster recovery without a major version change.
Relates #15971
Adding random shuffling of xContent to InnterHitBuilderTests shows
that the scriptFields are stored in order as a list internally although
they are an unordered json objects in the query dsl.
This changes the internal representation to a set and updates
serialization accordingly.
Currently we have a lot of methods left in QueryShardContext that
take parsers or BytesReference arguments to do some xContent
parsing on the shard. While this still seems necessary in some cases
(e.g. percolation, phrase suggester), the shard context should only
be concerned with generating lucene queries from QueryBuilders.
This change removes all of the parseX() methods in favour of two
public methods toQuery(QueryBuilder) and toFilter(QueryBuilder) that
either call the query builders toFilter() or toQuery() method and
move all code required for parsing out to the respective callers.
PathTrie has a constructor that allows for an arbitrary separtor and
wildcard, but this constructor is unused and internally we always use
'/' as the separator and '*' as the wildcard. There are no tests for the
case where the separator differs from the default separator and
wildcard. This commit removes this constructor and now all instances of
PathTrie have the default separator and wildcard.
This commit removes the method Strings#splitStringToArray and replaces
the call sites with invocations to String#split. There are only two
explanations for the existence of this method. The first is that
String#split is slightly tricky in that it accepts a regular expression
rather than a character to split on. This means that if s is a string,
s.split(".") does not split on the character '.', but rather splits on
the regular expression '.' which splits on every character (of course,
this is easily fixed by invoking s.split("\\.") instead). The second
possible explanation is that (again) String#split accepts a regular
expression. This means that there could be a performance concern
compared to just splitting on a single character. However, it turns out
that String#split has a fast path for the case of splitting on a single
character and microbenchmarks show that String#split has 1.5x--2x the
throughput of Strings#splitStringToArray. There is a slight behavior
difference between Strings#splitStringToArray and String#split: namely,
the former would return an empty array in cases when the input string
was null or empty but String#split will just NPE at the call site on
null and return a one-element array containing the empty string when the
input string is empty. There was only one place relying on this behavior
and the call site has been modified accordingly.
Mike McCandless
Ali Beyad
Daniel Mitterdorfer
Daniel Mitterdorfer
Robert Muir
Lee Hinman
Adrien Grand
Jason Tedor
Britta Weber
Christoph Büscher
Clinton Gormley
Olivier Bourgain
Zachary Tong
Robert Muir
Yannick Welsch
Gabriel Moskovicz
Colin Goodheart-Smithe
Alexander Kazakov
Johnny Lim
Jim Ferenczi
Ryan Ernst
Tal Levy
Chris Earle
Tanguy Leroux
Britta Weber
Nik Everett
Alexander Reelsen
Isabel Drost-Fromm