Commit Graph

7470 Commits

Author SHA1 Message Date
Przemysław Witek 28f68fa221
Make num_top_classes parameter's default value equal to 2 (#48119) (#48201) 2019-10-17 18:43:15 +02:00
James Rodewig 4c6d56bef0 [DOCS] Expand put mapping API docs examples (#47462) 2019-10-17 09:31:52 -04:00
James Rodewig b4bf47cce4 [DOCS] Document ingest pipeline and processor node stats (#47884) 2019-10-17 09:19:45 -04:00
Anton b54d817f1d [DOCS] Fix typo in intervals query docs (#48180) 2019-10-17 09:15:55 -04:00
Hendrik Muhs 06f54428c2 update internal index name in documentation warning (#48125) 2019-10-17 15:09:47 +02:00
István Zoltán Szabó ecd62934f6 [DOCS] Applies rename of transform related roles and privileges (#48123) 2019-10-17 10:24:56 +02:00
Martijn van Groningen a5fe69c344
Include enrich into the info api as feature (#48157)
This commit also fixes a bug, the enrich enabled setting
was not included in the list of settings.

Backport of #48109
2019-10-17 09:51:32 +02:00
Lee Hinman 5af66d79ef
Add SLM support to xpack usage and info APIs (#48149)
* Add SLM support to xpack usage and info APIs

This is a backport of #48096

This adds the missing xpack usage and info information into the
`/_xpack` and `/_xpack/usage` APIs. The output now looks like:

```
GET /_xpack/usage
{
  ...
  "slm" : {
    "available" : true,
    "enabled" : true,
    "policy_count" : 1,
    "policy_stats" : {
      "retention_runs" : 0,
      ...
    }
  }
```

and

```
GET /_xpack
{
  ...
  "features" : {
    ...
    "slm" : {
      "available" : true,
      "enabled" : true
    },
    ...
  }
}
```

Relates to #43663

* Fix missing license
2019-10-16 21:06:27 -06:00
David Woods 73f5fef678 Add a note to query_string docs around spaces in field names. (#47326) 2019-10-16 13:28:55 -07:00
James Rodewig 8677653c5b [DOCS] Reformat apostrophe token filter docs (#48076) 2019-10-16 08:51:14 -04:00
François-Clément Brossard f501a4b2b5 Clarify low watermark documentation (#48112)
Today the docs say that the low watermark has no effect on any shards that have
never been allocated, but this is confusing. Here "shard" means "replication
group" not "shard copy" but this conflicts with the "never been allocated"
qualifier since one allocates shard copies and not replication groups.

This commit removes the misleading words. A newly-created replication group
remains newly-created until one of its copies is assigned, which might be quite
some time later, but it seems better to leave this implicit.
2019-10-16 12:27:49 +01:00
Ian Danforth 28c1677341 [DOCS] Fix typo in percentile rank aggregation docs (#47247) 2019-10-15 15:56:45 -04:00
Wilder Pereira 8c73e215b2 [DOCS] Remove unneeded spaces from custom analyzer snippet (#47332) 2019-10-15 15:53:16 -04:00
James Rodewig 601a88bede [DOCS] Sort analyzers, tokenizers, and token filters alphabetically (#48068) 2019-10-15 15:47:25 -04:00
James Rodewig 3a7c2a4d17 [DOCS] Add `wait_for_completion` parm to execute enrich policy API docs (#48077) 2019-10-15 13:47:30 -04:00
Martijn van Groningen aff0c9babc
This commits merges (#48040) the enrich-7.x feature branch,
which is backport merge and adds a new ingest processor, named enrich processor,
that allows document being ingested to be enriched with data from other indices.

Besides a new enrich processor, this PR adds several APIs to manage an enrich policy.
An enrich policy is in charge of making the data from other indices available to the enrich processor in an efficient manner.

Related to #32789
2019-10-15 17:31:45 +02:00
James Rodewig 05b3de0110 [DOCS] Expand `updated` response parm in reindex API docs (#48008) 2019-10-15 11:17:09 -04:00
Marios Trivyzas 7fddf198b7 SQL: Implement DATEDIFF function (#47920)
Implement DATEDIFF/TIMESTAMPDIFF function as per the MS-SQL spec:
https://docs.microsoft.com/en-us/sql/t-sql/functions/datediff-transact-sql?view=sql-server-2017
which allows a user to substract two date/datetime fields and return the
difference in the date/time unit specified.

Closes: #47919
(cherry picked from commit 745699f38dc8222670ffd65b66df33b5da39040b)
2019-10-15 15:12:11 +02:00
James Rodewig a61d12afea [DOCS] Add #46860 to 7.4 release notes (#48018) 2019-10-15 08:48:27 -04:00
Marios Trivyzas 6589617a51
SQL: Fix arg verification for DateAddProcessor (#48041)
Previously, the safety check for the 2nd argument of the DateAddProcessor was
restricting it to Integer which was wrong since we allow all non-rational
numbers, so it's changed to a Number check as it's done in other cases.

Enhanced some tests regarding the check for an integer (non-rational
argument).

(cherry picked from commit 0516b6eaf5eb98fa5bd087c3fece80139a6b118e)
2019-10-15 12:52:11 +02:00
Martijn van Groningen 31e41d4ac2
fixed invalid reference 2019-10-15 10:45:35 +02:00
Tanguy Leroux fe15d9f784
Add documentation for Pause/Resume Auto-Follower APIs (#47985) (#48034)
Relates #47510

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-10-15 08:38:49 +02:00
David Roberts 984323783e
[ML][7.x] Add lazy assignment job config option (#47993)
This change adds:

- A new option, allow_lazy_open, to anomaly detection jobs
- A new option, allow_lazy_start, to data frame analytics jobs

Both work in the same way: they allow a job to be
opened/started even if no ML node exists that can
accommodate the job immediately. In this situation
the job waits in the opening/starting state until ML
node capacity is available. (The starting state for data
frame analytics jobs is new in this change.)

Additionally, the ML nightly maintenance tasks now
creates audit warnings for ML jobs that are unassigned.
This means that jobs that cannot be assigned to an ML
node for a very long time will show a yellow warning
triangle in the UI.

A final change is that it is now possible to close a job
that is not assigned to a node without using force.
This is because previously jobs that were open but
not assigned to a node were an aberration, whereas
after this change they'll be relatively common.
2019-10-15 06:55:11 +01:00
Martijn van Groningen cc4b6c43b3
Merge remote-tracking branch 'es/7.x' into enrich-7.x 2019-10-15 07:23:47 +02:00
Gordon Brown 300ddfa3c1
SLM Start/Stop HLRC and docs (#47966)
This commit adds HLRC support and documentation for the SLM Start and
Stop APIs, as well as updating existing documentation where appropriate.

This commit also ensures that the SLM APIs are properly included in the
HLRC documentation.
2019-10-14 16:56:31 -06:00
Martijn van Groningen 7fc9198d46
Change how `max_matches` affects `target_field` option. (#47982)
Prior to this change the `target_field` would always be a json array
field in the document being ingested. This to take into account that
multiple enrich documents could be inserted into the `target_field`.

However the default `max_matches` is `1`. Meaning that by default
only a single enrich document would be added to `target_field` json
array field.

This commit changes this; if `max_matches` is set to `1` then the single
document would be added as a json object to the `target_field` and
if it is configured to a higher value then the enrich documents will be
added as a json array (even if a single enrich document happens to be
enriched).
2019-10-14 21:09:48 +02:00
Martijn van Groningen 7cc73f6193
Add HLRC support for enrich execute policy API (#47991)
This PR also includes HLRC docs for the enrich stats api.

Relates to #32789
2019-10-14 19:55:48 +02:00
James Rodewig 170266765b [DOCS] Reformat docs for several snapshot lifecycle policy APIs (#47998) 2019-10-14 12:30:30 -04:00
Tanguy Leroux e4ea8b46b6
Add Pause/Resume Auto-Follower APIs to High Level REST Client (#48004)
This commit adds support for Pause/Resume Auto-Follower APIs 
to the HLRC, with the documentation.

Relates #47510
2019-10-14 18:25:53 +02:00
David Roberts 1ca25bed38
[ML][7.x] Add option to stop datafeed that finds no data (#47995)
Adds a new datafeed config option, max_empty_searches,
that tells a datafeed that has never found any data to stop
itself and close its associated job after a certain number
of real-time searches have returned no data.

Backport of #47922
2019-10-14 17:19:13 +01:00
James Rodewig 5f3ef2e09c [DOCS] Remove unsupported `local` and `master_timeout` parms from cat API docs (#47933) 2019-10-14 11:39:31 -04:00
Martijn van Groningen d4901a71d7
Merge remote-tracking branch 'es/7.x' into enrich-7.x 2019-10-14 10:27:17 +02:00
Tanguy Leroux 742fa818b8
Add Pause/Resume Auto Follower APIs (#47510) (#47904)
This commit adds two APIs that allow to pause and resume
CCR auto-follower patterns:

// pause auto-follower
POST /_ccr/auto_follow/my_pattern/pause

// resume auto-follower
POST /_ccr/auto_follow/my_pattern/resume

The ability to pause and resume auto-follow patterns can be
useful in some situations, including the rolling upgrades of
cluster using a bi-directional cross-cluster replication scheme
(see #46665).

This commit adds a new active flag to the AutoFollowPattern
and adapts the AutoCoordinator and AutoFollower classes so
that it stops to fetch remote's cluster state when all auto-follow
patterns associate to the remote cluster are paused.

When an auto-follower is paused, remote indices that match the
pattern are just ignored: they are not added to the pattern's
followed indices uids list that is maintained in the local cluster
state. This way, when the auto-follow pattern is resumed the
indices created in the remote cluster in the meantime will be
picked up again and added as new following indices. Indices
created and then deleted in the remote cluster will be ignored
as they won't be seen at all by the auto-follower pattern at
resume time.

Backport of #47510 for 7.x
2019-10-13 09:22:51 +02:00
Yogesh Gaikwad ac209c142c
Remove uniqueness constraint for API key name and make it optional (#47549) (#47959)
Since we cannot guarantee the uniqueness of the API key `name` this commit removes the constraint and makes this field optional.

Closes #46646
2019-10-12 22:22:16 +11:00
James Rodewig 256102042b
[DOCS] Reformat get snapshot lifecycle policy API docs (#47827) (#47930) 2019-10-11 13:00:48 -04:00
James Rodewig aa5689f5b3
[DOCS] Reformat put snapshot lifecycle policy API docs (#47811) (#47925) 2019-10-11 11:28:38 -04:00
David Turner ba62eb3dce Allow truncation of clean translog (#47866)
Today the `elasticsearch-shard remove-corrupted-data` tool will only truncate a
translog it determines to be corrupt. However there may be other cases in which
it is desirable to truncate the translog, for instance if an operation in the
translog cannot be replayed for some reason other than corruption. This commit
adds a `--truncate-clean-translog` option to skip the corruption check on the
translog and blindly truncate it.
2019-10-11 15:48:12 +01:00
James Rodewig 58289a857d
[DOCS] Reformat reload search analyzers API docs (#47824) (#47907) 2019-10-11 09:40:49 -04:00
Przemysław Witek d210bfa888
[7.x] Add MlClientDocumentationIT tests for classification. (#47569) (#47896) 2019-10-11 10:19:55 +02:00
Matt Braymer-Hayes d315726c23 [DOCS] Fix gcloud command typo in discovery-gce.asciidoc. (#47312) 2019-10-10 14:21:46 -04:00
Marios Trivyzas 59b3294bc9 SQL: Implement DATEADD function (#47747)
Implement DATEADD/TIMESTAMPADD function as per the MS-SQL spec:
https://docs.microsoft.com/en-us/sql/t-sql/functions/dateadd-transact-sql?view=sql-server-2017
which allows a user to add/subtract specified number of specified units
to/from a date/datetime field/expression.

Closes: #47746
(cherry picked from commit e624bc281bebb4bbe0b0c2e0a8cbc712e50097a8)
2019-10-10 16:22:13 +02:00
James Rodewig a8e027551a
[DOCS] Document `bytes` and `time` params in cat API docs (#47672) (#47854) 2019-10-10 09:48:40 -04:00
Martijn van Groningen 102016d571
Merge remote-tracking branch 'es/7.x' into enrich-7.x 2019-10-10 14:44:05 +02:00
István Zoltán Szabó 9eac8bf2a8 [DOCS] Adds supported fields section to the PUT DFA API description (#47842) 2019-10-10 12:42:54 +02:00
David Turner ecb20ebc6c More bootstrap docs tweaks (#47809)
Clarifies not to set `cluster.initial_master_nodes` on nodes that are joining
an existing cluster.

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-10-10 09:55:30 +01:00
Jim Ferenczi dc39196ea4 Fix tag in the search request timeout option docs (#47776)
and add missing parentheses `search_timeout` param
2019-10-10 10:35:44 +02:00
Martijn van Groningen aace42d38d
Add HLRC support for enrich stats API (#47306)
This PR also includes HLRC docs for the enrich stats api.

Relates to #32789
2019-10-10 09:08:29 +02:00
István Zoltán Szabó 6f4b7e9a7f [DOCS] Extends the analyzed_fields description in the PUT DFA API docs (#47791) 2019-10-09 18:14:58 +02:00
James Rodewig 65f8294378 [DOCS] Add docs for `geo_match` enrich policy type (#47745) 2019-10-09 09:02:52 -04:00
Florian Kelbert 2abd9d53b6 [DOCS] Correct split API request format (#47774) 2019-10-09 08:28:41 -04:00