* master:
ingest: Add ignore_missing property to foreach filter (#22147) (#31578)
Fix a formatting issue in the docvalue_fields documentation. (#31563)
reduce log level at gradle configuration time
[TEST] Close additional clients created while running yaml tests (#31575)
Docs: Clarify sensitive fields watcher encryption (#31551)
Watcher: Remove never executed code (#31135)
Add support for switching distribution for all integration tests (#30874)
Improve robustness of geo shape parser for malformed shapes (#31449)
QA: Create xpack yaml features (#31403)
Improve test times for tests using `RandomObjects::addFields` (#31556)
[Test] Add full cluster restart test for Rollup (#31533)
Enhance thread context uniqueness assertion
[DOCS] Fix heading format errors (#31483)
fix writeIndex evaluation for aliases (#31562)
Add x-opaque-id to search slow logs (#31539)
Watcher: Fix put watch action (#31524)
Add package pre-install check for java binary (#31343)
Reduce number of raw types warnings (#31523)
Migrate scripted metric aggregation scripts to ScriptContext design (#30111)
turn GetFieldMappingsResponse to ToXContentObject (#31544)
Close xcontent parsers (partial) (#31513)
Ingest Attachment: Upgrade Tika to 1.18 (#31252)
TEST: Correct the assertion arguments order (#31540)
* Migrate scripted metric aggregation scripts to ScriptContext design #29328
* Rename new script context container class and add clarifying comments to remaining references to params._agg(s)
* Misc cleanup: make mock metric agg script inner classes static
* Move _score to an accessor rather than an arg for scripted metric agg scripts
This causes the score to be evaluated only when it's used.
* Documentation changes for params._agg -> agg
* Migration doc addition for scripted metric aggs _agg object change
* Rename "agg" Scripted Metric Aggregation script context variable to "state"
* Rename a private base class from ...Agg to ...State that I missed in my last commit
* Clean up imports after merge
* master:
Add get field mappings to High Level REST API Client (#31423)
[DOCS] Updates Watcher examples for code testing (#31152)
TEST: Add bwc recovery tests with synced-flush index
[DOCS] Move sql to docs (#31474)
[DOCS] Move monitoring to docs folder (#31477)
Core: Combine doExecute methods in TransportAction (#31517)
IndexShard should not return null stats (#31528)
fix repository update with the same settings but different type (#31458)
Fix Mockito trying to mock IOException that isn't thrown by method (#31433) (#31527)
Node selector per client rather than per request (#31471)
Core: Combine messageRecieved methods in TransportRequestHandler (#31519)
Upgrade to Lucene 7.4.0. (#31529)
[ML] Add ML filter update API (#31437)
Allow multiple unicast host providers (#31509)
Avoid deprecation warning when running the ML datafeed extractor. (#31463)
REST high-level client: add simulate pipeline API (#31158)
Get Mapping API to honour allow_no_indices and ignore_unavailable (#31507)
[PkiRealm] Invalidate cache on role mappings change (#31510)
[Security] Check auth scheme case insensitively (#31490)
In NumberFieldType equals and hashCode, make sure that NumberType is taken into account. (#31514)
[DOCS] Fix REST tests in SQL docs
[DOCS] Add code snippet testing in more ML APIs (#31339)
Core: Remove ThreadPool from base TransportAction (#31492)
[DOCS] Remove fixed file from build.gradle
Rename createNewTranslog to fileBasedRecovery (#31508)
Test: Skip assertion on windows
[DOCS] Creates field and document level security overview (#30937)
[DOCS] Significantly improve SQL docs
[DOCS] Move migration APIs to docs (#31473)
Core: Convert TransportAction.execute uses to client calls (#31487)
Return transport addresses from UnicastHostsProvider (#31426)
Ensure local addresses aren't null (#31440)
Remove unused generic type for client execute method (#31444)
Introduce http and tcp server channels (#31446)
Since #31251, we use the default distribution to test docs. With this
distribution, the cat thread-pool response will include the ccr
thread-pool. This commit adjusts /_cat/thread_pool doc to include ccr.
Relates #31251
A link to the ip_range datatype page provides a way for newer users to know
it exists if they land directly on the ip datatype page first via a search.
The `multiplexer` filter emits multiple tokens at the same position, each
version of the token haivng been passed through a different filter chain.
Identical tokens at the same position are removed.
This allows users to, for example, index lowercase and original-case tokens,
or stemmed and unstemmed versions, in the same field, so that they can search
for a stemmed term within x positions of an unstemmed term.
Folks tend to want to be able to make a single `_reindex` call to
migrate many indices. You *can* do that and we even have an example of
how to do that in the docs but it isn't always a good idea. This change
adds some advice to the docs: generally you want to make one reindex
call per index.
Closes#22920
This switches the docs tests from the `oss-zip` distribution to the
`zip` distribution so they have xpack installed and configured with the
default basic license. The goal is to be able to merge the
`x-pack/docs` directory into the `docs` directory, marking the x-pack
docs with some kind of marker. This is the first step in that process.
This also enables `-Dtests.distribution` support for the `docs`
directory so you can run the tests against the `oss-zip` distribution
with something like
```
./gradlew -p docs check -Dtests.distribution=oss-zip
```
We can set up Jenkins to run both.
Relates to #30665
The other metric aggregations (min/max/etc) return `null` as their XContent value and string when nothing was computed (due to empty/missing fields). Percentiles and Percentile Ranks, however, return `NaN `which is inconsistent and confusing for the user. This fixes the inconsistency by making the aggs return `null`. This applies to both the numeric value and the "as string" value.
Note: like the metric aggs, this does not change the value if fetched directly from the percentiles object, which will return as `NaN`/`"NaN"`. This only changes the XContent output.
While this is a bugfix, it still breaks bwc in a minor way as the response changes from prior version.
Closes#29066
This commit adds the is-write-index flag for aliases.
It allows requests to set the flag, and responses to display the flag.
It does not validate and/or affect any indexing/getting/updating behavior
of Elasticsearch -- this will be done in a follow-up PR.
With #29331 we added support for the cluster health API to the
high-level REST client. The transport client does not support the level
parameter, and it always returns all the info needed for shards level
rendering. We have maintained that behaviour when adding support for
cluster health to the high-level REST client, to ease migration, but the
correct thing to do is to default the high-level REST client to
`cluster` level, which is the same default as when going through the
Elasticsearch REST layer.
This commit removes all the API methods that accept a `Header` varargs
argument, in favour of the newly introduced API methods that accept a
`RequestOptions` argument.
Relates to #31069
This adds a thread interrupter that allows us to encapsulate calls to org.joni.Matcher#search()
This method can hang forever if the regex expression is too complex.
The thread interrupter in the background checks every 3 seconds whether there are threads
execution the org.joni.Matcher#search() method for longer than 5 seconds and
if so interrupts these threads.
Joni has checks that that for every 30k iterations it checks if the current thread is interrupted and
if so returns org.joni.Matcher#INTERRUPTED
Closes#28731
With `max_concurrent_shard_requests` we used to throttle / limit
the number of concurrent shard requests a high level search request
can execute per node. This had several problems since it limited the
number on a global level based on the number of nodes. This change
now throttles the number of concurrent requests per node while still
allowing concurrency across multiple nodes.
Closes#31192
Adds support for `ignore_unmapped` parameter in geo distance sorting,
which is functionally equivalent to specifying an `unmapped_type` in
the field sort.
Closes#28152
This field is similar to the `feature` field but is better suited to index
sparse feature vectors. A use-case for this field could be to record topics
associated with every documents alongside a metric that quantifies how well
the topic is connected to this document, and then boost queries based on the
topics that the logged user is interested in.
Relates #27552
By default span_multi query will limit term expansions = boolean max clause.
This will limit high heap usage in case of high cardinality term
expansions. This applies only if top_terms_N is not used in inner multi
query.
