Various classes had some code that was not used and is not going to be needed so this change cleans up those classes so we don’t have dead code hanging around
In case of a disjunction query with both range and term based clauses and
msm specified, the query analyzer needs to also reduce the msn if a range
based clause for the same field is encountered. This did not happen.
Instead of fixing this bug the logic has been simplified to just set a
percolator query's msm to 1 if a disjunction contains range clauses and
msm on disjunction has been specified. The logic would otherwise just get
to complex and the performance gain isn't that much for this kind of
percolator queries.
In case a percolator query has clauses that have duplicate terms or ranges then
for disjunction clauses with a minimum should match the query extraction of the
clause with the lowest msm should be used and for conjunction queries query
extractions wiht duplicate terms/ranges the msn should be ignored. If this
is not done then percolator queries that should match never match.
Example percolator query: value1 OR value2 OR value2 OR value3 OR value3 OR value3 OR value4 OR value5 (msm set to 3)
In the above example query the extracted msm would be 3
Example document1: value1 value2 value3
With the msm and extracted terms this would match and is expected behaviour
Example document2: value3
This document should match too (value3 appears in 3 clauses), but with msm set to 3 and the fact
that fact that only distinct values are indexed in extracted terms field this document would
Also added another random duel test.
Closes#29393
The force-merge is an a TODO state due to the
unresolved issue around best_compression.
- updated ReadOnlyStep with tests
- implemented an update to the ForceMergeAction
- added UpdateBestCompressionSettingsStep
- added tests for SegmentCountStep
* Remove copy-pasted code
We had two instances of copy-pasted code with a bad license from
another website. The code was doing something rather simple, and
that functionality already exists within junit. This PR simply leverages
the junit functionality.
For the user cache, the crypt option rounds are actually the log2 of the number
of rounds. This commits updates the documentation to reflect this.
Original commit: elastic/x-pack-elasticsearch@d3cc2b7f29
`BaseRandomBinaryDocValuesRangeQueryTestCase.testRandomBig` should only run with
nightly tests. It doesn't make sense to make it part of every test run.
`UUIDTests` had a slow test for compression, which I made a bit faster by
decreasing the number of indexed docs.
This change tries to simplify the extraction logic of boolean queries by
concentrating the logic into two methods: one that merges results for
conjunctions, and another one for disjunctions. Other concerns, like the impact
of prohibited clauses or how an `UnsupportedQueryException` should be treated
are applied on top of those two methods.
This is mostly a code reorganization, it doesn't change the result of query
extraction except in the case that a query both has required clauses and a
minimum number of `SHOULD` clauses that is greater than 1, which we now
rewrite into a pure conjunction. For instance `(+A B C)~1` is rewritten into
`(+A +(B C))` prior to extraction.
With the change of requiring to configure account settings properly by
using affix settings, we forgot another special snowflake, namely the
configuration of mail properties, which can be arbitrary in the
configuration. Those properties are used when an email is sent.
This commit adds a few (but not all of those) options back and removes
the link in the documentation to refer to all of those settings.
Some settings are useless, as they only change the execution
expectations when a mail is sent, which the watch has control over.
The following settings are supported
* smtp.{host,port,user,password}
* smtp.auth
* smtp.starttls.{enable,required}
* smtp.{timeout,connection_timeout,write_timeout}
* smtp.{local_address,local_port}
* smtp.send_partial
* smtp.wait_on_quit
relates elastic/x-pack-elasticsearch#4048
Original commit: elastic/x-pack-elasticsearch@39d5624710
This commit fixes the formatting of the values in the composite
aggregation response. `date` fields should return timestamp as longs
when used in a `terms` source and `ip` fields should always be formatted as strings.
This commit also fixes the parsing of the `after` key for these field types.
Finally, this commit disables the index optimization for the `ip` field and any source that provides a `missing` value.
This commit simplifies the invocations to
Translog#closeOnTragicEvent. This method already catches all possible
exceptions and suppresses the non-AlreadyClosedExceptions into the
exception that triggered the invocation. Therefore, there is no need for
callers to do this same logic (which would never execute).
Adds `value_count` as one of the accepted metrics. The caveat is that
it only accepts numeric values for two reasons:
- Job validation at creation makes sure all metrics are numeric fields.
Changing this would require new syntax (or disallowing anything but
value_count on mixed fields)
- when `toBuilders()` is called, we have to supply a ValueSource to
the ValueCountBuilder, and we don't know what the field type is at that
time.
These are both fixable, but relatively more involved. I think numeric-only
is a reasonable limitation to start with
Original commit: elastic/x-pack-elasticsearch@270f24c8bf
* Move Streams.copy into elasticsearch-core and make a multi-release jar
This moves the method `Streams.copy(InputStream in, OutputStream out)` into the
`elasticsearch-core` project (inside the `o.e.core.internal.io` package). It
also makes this class into a multi-release class where the Java 9 equivalent
uses `InputStream#transferTo`.
This is a followup from
https://github.com/elastic/elasticsearch/pull/29300#discussion_r178147495
* Move ObjectParser into the x-content lib
This moves `ObjectParser`, `AbstractObjectParser`, and
`ConstructingObjectParser` into the libs/x-content dependency. This decoupling
allows them to be used for parsing for projects that don't want to depend on the
entire Elasticsearch jar.
Relates to #28504
* Move Tuple into elasticsearch-core
This allows us to use Tuple from other projects that don't want to rely on the
entire Elasticsearch jar.
I have also added very simple tests, since there were none.
Relates tangentially to #28504
Today we close the translog write tragically if we experience any I/O
exception on a write. These tragic closes lead to use closing the
translog and failing the engine. Yet, there is one case that is missed
which is when we touch the write channel during a read (checking if
reading from the writer would put us past what has been flushed). This
commit addresses this by closing the writer tragically if we encounter
an I/O exception on the write channel while reading. This becomes
interesting when we consider that this method is invoked from the engine
through the translog as part of getting a document from the
translog. This means we have to consider closing the translog here as
well which will cascade up into us finally failing the engine.
Note that there is no semantic change to, for example, primary/replica
resync and recovery. These actions will take a snapshot of the translog
which syncs the translog to disk. If an I/O exception occurs during the
sync we already close the writer tragically and once we have synced we
do not ever read past the position that was synced while taking the
snapshot.
Currently the ranking evaluation API doesn't support many of the
standard parameters of the search API. Some of these make sense, like
adding support for the common indices options parameters, which this
change adds.
Add a check to the migration assistant to warn about the renaming of
`delimited_payload_filter` to `delimited_payload`. This should still
word for old indices from 7.0 on but will throw an error for newly
created indices and the user should be warned about it when running the
migration checks.
Original commit: elastic/x-pack-elasticsearch@5d55e4e499
* Fixes query_string query equals timezone check
This change fixes a bug where two `QueryStringQueryBuilder`s were found
to be equal if they had the same timezone set even if the query string
in the builders were different
Closes#29403
* Adds mutate function to QueryStringQueryBuilderTests
* iter
This change moves the -ea and -esa options that enable assertions for
test nodes before the cluster-specific JVM arguments on the Java command
line. This opens up the possibility for the cluster-specific JVM
arguments to disable assertions for one particular package or class,
which can be useful in BWC testing where incorrect assertions cannot be
removed from released versions of the product.
Fixes instances of
- Equals methods without type check
- Equals methods where the field of `this` was compared to the same
field of `this` instead of the `that` object that is compared to
This commit adds a new fixture that emulates an
Azure Storage service in order to improve the
existing integration tests. This is very similar
to what has been made for Google Cloud Storage
in #28788 and for Amazon S3 in #29296, and it
would have helped a lot to catch bugs like #22534.
I found the following bugs:
- The 6.0 logic for conjunctions didn't work when there were only `match_all`
queries in MUST/FILTER clauses as they didn't propagate the `matchAllDocs`
flag.
- Some queries still had the same issue as `BooleanQuery` used to have with
duplicate terms (see #28353), eg. `MultiPhraseQuery`.
Closes#29376
The repository-gcs unit tests rely on the GoogleCloudStorageTestServer
but it would be better if they rely on a mocked Storage client instead.
That would also help to extract the GoogleCloudStorageFixture and the
GoogleCloudStorageTestServer classes in a QA third party project.
Closes#28960
We specify an alias for signing key, but when we just have
a single key in key store this is an additional setting which
is annoying. This PR addresses this issue by making it optional.
- Changes in SamlRealmSettings to make signing/encryption
key alias optional
- Checks if none of the keys are useful for given operation
signing or encryption throws an error.
- Checks for no of aliases in key-store, if more than one and alias
is not specified throws error.
- If an alias is not specified and there is just one alias in
keystore then use it as the credential.
- Unit Tests
Note: A side effect of this change the above-mentioned behavior is
it's also applicable for encryption keys currently, but it is going
to change when fixing elastic/x-pack-elasticsearch#3980 for supporting multiple encryption keys.
relates elastic/x-pack-elasticsearch#3981
Original commit: elastic/x-pack-elasticsearch@2b5af1d8a8
Colin Goodheart-Smithe
D Pinto
Martijn van Groningen
Tal Levy
tomcallahan
Igor Motov
Lisa Cawley
Lee Hinman
David Kyle
Adrien Grand
Alexander Reelsen
Jim Ferenczi
yangyang.liu
Jason Tedor
Christoph Büscher
Zachary Tong
Christoph Büscher
Van0SS
Tanguy Leroux
David Roberts
Dimitris Athanasiou
Menno Oudshoorn
Yogesh Gaikwad