* es/master: (30 commits)
[Docs] Fix Java Api index administration usage (#28133)
Fix eclipse build. (#28236)
Never return null from Strings.tokenizeToStringArray (#28224)
Fallback to TransportMasterNodeAction for cluster health retries (#28195)
[Docs] Changes to ingest.asciidoc (#28212)
TEST: Update logging for testAckedIndexing
[GEO] Add WKT Support to GeoBoundingBoxQueryBuilder
Painless: Add whitelist extensions (#28161)
Fix daitch_mokotoff phonetic filter to use the dedicated Lucene filter (#28225)
Avoid doing redundant work when checking for self references. (#26927)
Fix casts in HotThreads. (#27578)
Ignore the `-snapshot` suffix when comparing the Lucene version in the build and the docs. (#27927)
Allow update of `eager_global_ordinals` on `_parent`. (#28014)
Fix NPE on composite aggregation with sub-aggregations that need scores (#28129)
`MockTcpTransport` to connect asynchronously (#28203)
Fix synonym phrase query expansion for cross_fields parsing (#28045)
Introduce elasticsearch-core jar (#28191)
#28218: Update the Lucene version for 6.2.0 after backport
upgrade to lucene 7.2.1 (#28218)
[Docs] Fix an error in painless-types.asciidoc (#28221)
...
The Java API documentation for index administration currenty is wrong because
the PutMappingRequestBuilder#setSource(Object... source) and
CreateIndexRequestBuilder#addMapping(String type, Object... source) methods
delegate to methods that check that the input arguments are valid key/value
pairs:
https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-admin-indices.html
This changes the docs so the java api code examples are included from
documentation integration tests so we detect compile and runtime issues earlier.
Closes#28131
This method has a different contract than all the other methods in this class, returning null instead of an empty array when receiving a null input. While switching over some methods from delimitedListToStringArray to this method tokenizeToStringArray, this resulted in unexpected nulls in some places of our code.
Relates #28213
Currently we test all maps, arrays or iterables. However, in the case that maps
contain sub maps for instance, we will test the sub maps again even though the
work has already been done for the top-level map.
Relates #26907
A bug introduced in #24407 currently prevents `eager_global_ordinals` from
being updated. This new approach should fix the issue while still allowing
mapping updates to not specify the `_parent` field if it doesn't need
updating, which was the goal of #24407.
The composite aggregation defers the collection of sub-aggregations to a second pass that visits documents only if they
appear in the top buckets. Though the scorer for sub-aggregations is not set on this second pass and generates an NPE if any sub-aggregation
tries to access the score. This change creates a scorer for the second pass and makes sure that sub-aggs can use it safely to check the score of
the collected documents.
The method `initiateChannel` on `TcpTransport` is explicit in that
channels can be connect asynchronously. All production implementations
do connect asynchronously. Only the blocking `MockTcpTransport`
connects in a synchronous manner. This avoids testing some of the
blocking code in `TcpTransport` that waits on connections to complete.
Additionally, it requires a more extensive method signature than
required for other transports.
This commit modifies the `MockTcpTransport` to make these connections
asynchronously on a different thread. Additionally, it simplifies that
`initiateChannel` method signature.
* Fix synonym phrase query expansion for cross_fields parsing
The `cross_fields` mode for query parser ignores phrase query generated by multi-word synonyms.
In such case only the first field of each analyzer group is kept. This change fixes this issue
by expanding the phrase query for each analyzer group to **all** fields using a disjunction max query.
This is related to #27933. It introduces a jar named elasticsearch-core
in the lib directory. This commit moves the JarHell class from server to
elasticsearch-core. Additionally, PathUtils and some of Loggers are
moved as JarHell depends on them.
* Adds metadata to rewritten aggregations
Previous to this change, if any filters in the filters aggregation were rewritten, the rewritten version of the FiltersAggregationBuilder would not contain the metadata form the original. This is because `AbstractAggregationBuilder.getMetadata()` returns an empty map when not metadata is set.
Closes#28170
* Always set metadata when rewritten
As a replica always keeps a safe commit and starts peer-recovery with
that commit; file-based recovery only happens if new operations are
added to the primary and the required translog is not fully retained. In
the test, we tried to produce this condition by flushing a new commit in
order to trim all translog. However, if the new global checkpoint is not
persisted yet, we will keep two commits and not trim translog. This
commit tightens the file-based condition in the test by waiting for the
global checkpoint persisted properly on the new primary before flushing.
Close#28209
Relates #28181
We introduced a new option `createNewTranslog` in #28181. However, we
named that parameter as deleteLocalTranslog in other places. This commit
makes sure to have a consistent naming in these places.
Relates #28181
The global checkpoint should be assigned to unassigned rather than 0. If
a single document is indexed and the global checkpoint is initialized
with 0, the first commit is safe which the test does not suppose.
Relates #28038
Today a replica starts a peer-recovery with the last commit. If the last
commit is not a safe commit, a replica will immediately fallback to the
file based sync which is more expensive than the sequence based
recovery. This commit modifies the peer-recovery in replica to start
with a safe commit. Moreover we can keep the existing translog on the
target if the recovery is sequence based recovery.
Relates #10708
We are targeting to always have a safe index once the recovery is done.
This invariant does not hold if the translog is manually truncated by
users because the truncate translog cli resets the global checkpoint to
unassigned. This commit assigns the global checkpoint to the max_seqno
of the last commit when truncating translog. We can only safely do it
because the truncate translog command will generate a new history uuid
for that shard. With a new history UUID, sequence-based recovery between
that shard and other old shards will be disabled.
Relates #28181
* master:
Fix lock accounting in releasable lock
Add ability to associate an ID with tasks (#27764)
[DOCS] Removed differencies between text and code (#27993)
text fixes (#28136)
Update getting-started.asciidoc (#28145)
[Docs] Spelling fix in painless-getting-started.asciidoc (#28187)
Fixed the cat.health REST test to accept 4ms, not just 4.0ms (#28186)
Do not keep 5.x commits once having 6.x commits (#28188)
Releasble locks hold accounting on who holds the lock when assertions
are enabled. However, the underlying lock can be re-entrant yet we mark
the lock as not held by the current thread as soon as the releasable is
closed. For a re-entrant lock this is not right because the thread could
have entered the lock multiple times. Instead, we have to count how many
times the thread has entered the lock and only mark the lock as not held
by the current thread when the counter reaches zero.
Relates #28202
Currently we keep a 5.x index commit as a safe commit until we have a
6.x safe commit. During that time, if peer-recovery happens, a primary
will send a 5.x commit in file-based sync and the recovery will even
fail as the snapshotted commit does not have sequence number tags.
This commit updates the combined deletion policy to delete legacy
commits if there are 6.x commits.
Relates #27606
Relates #28038
* master: (43 commits)
Rename core module to server (#28180)
upgraded jna from 4.4.0-1 to 4.5.1 (#28183)
[TEST] Do not call RandomizedTest.scaledRandomIntBetween from multiple threads
Primary send safe commit in file-based recovery (#28038)
[Docs] Correct response json in rank-eval.asciidoc
Add scroll parameter to _reindex API (#28041)
Include all sentences smaller than fragment_size in the unified highlighter (#28132)
Modifies the JavaAPI docs related to AggregationBuilder
[Docs] Improvements in script-fields.asciidoc (#28174)
[Docs] Remove Kerberos/SPNEGO Shield plugin (#28019)
Ignore null value for range field (#27845) (#28116)
Fix environment variable substitutions in list setting (#28106)
docs: Replaces indexed script java api docs with stored script api docs
test: ensure we endup with a single segment
Make sure that we don't detect files as maven coordinate when installing a plugin (#28163)
[Tests] temporary disable meta plugin rest tests #28163
meta-plugin should install bin and config at the top level (#28162)
Painless: Add public member read/write access test. (#28156)
Docs: Clarify password protection support with keystore (#28157)
[Docs] fix plugin properties inclusion for plugins authors
...
Martijn van Groningen
Christoph Büscher
Yannick Welsch
Nhat Nguyen
Nicholas Knize
Adrien Grand
Jim Ferenczi
Tim Brooks
Jim Ferenczi
Colin Goodheart-Smithe
Igor Motov
Jason Tedor