Commit Graph

6129 Commits

Author SHA1 Message Date
Gordon Brown bb51cdb6de
Add HLRC docs for Get Lifecycle Policy (#35612)
Adds docs for the Get Lifecycle Policy API to the HLRC documentation.
2018-11-16 18:16:30 -07:00
Gordon Brown cce9648f9d
Align RolloverStep's name with other step names (#35655)
RolloverStep previously had a name of "attempt_rollover", which was
inconsistent with all other step names due it its use of an underscore
instead of a dash.
2018-11-16 17:42:48 -07:00
Julie Tibshirani c6a0904e0e
Deprecate types in count and msearch. (#35421)
* Deprecate types in count requests.
* Move RestCountAction to the 'search' package.
* Deprecate types in multi search requests.
* Add tests for types deprecation in the _search endpoint.
2018-11-16 13:04:43 -08:00
debadair fc63ea5357
[DOCS] Add beta warning to ILM pages. (#35571) 2018-11-16 10:49:55 -08:00
Julie Tibshirani 40ba4de5e6
Deprecate types in validate query requests. (#35575) 2018-11-16 08:59:04 -08:00
Martijn van Groningen 797cffbb68
[TEST] Replace fields in response with actual values
Relates to #35496
2018-11-16 10:18:14 +01:00
Przemyslaw Gomulka c6e2634bf8
Docs: HLRC: refactor bulk, migrate and reindex apis (#35413)
fix typos and refactor to DRY up documentation for bulk, reindex and migration apis
relates #35345
2018-11-16 08:58:13 +01:00
Tim Vernum 87a8b99724
HLRC: Add "_has_privileges" API to Security Client (#35479)
This adds the "hasPrivileges()" method to SecurityClient, including
request, response & async variant of the method.

Also includes API documentation.
2018-11-16 13:52:06 +11:00
Jun Ohtani 0cdfc4cd0a
[Doc] Add clarification to boolean query (#32575)
It isn't very clear how boosting query works.
Add explanation of positive/negative query.
2018-11-16 11:45:32 +09:00
Gordon Brown 3883e9bf4c
Split RolloverStep into Wait and Action steps (#35524)
RolloverAction will now periodically check the rollover conditions using
the Rollover API with the dry_run option as an AsyncWaitStep, then run
the rollover itself by calling the Rollover API with no conditions,
which will always roll over, as an AsyncActionStep. This will resolve
race condition issues in policies using RolloverAction.
2018-11-15 17:11:31 -07:00
Mayya Sharipova a8833b097b
Painless Context Doc: Add min should match example (#35423) 2018-11-15 15:10:49 -05:00
Mayya Sharipova 501c03e529
Painless Context Doc: Add filter context example (#35305) 2018-11-15 14:33:34 -05:00
Benjamin Trent f7ada9b29b
Add delayed datacheck to the datafeed job runner (#35387)
* ML: Adding missing datacheck to datafeedjob

* Adding client side and docs

* Making adjustments to validations

* Making values default to on, having more sensible limits

* Intermittent commit, still need to figure out interval

* Adjusting delayed data check interval

* updating docs

* Making parameter Boolean, so it is nullable

* bumping bwc to 7 before backport

* changing to version current

* moving delayed data check config its own object

* Separation of duties for delayed data detection

* fixing checkstyles

* fixing checkstyles

* Adjusting default behavior so that null windows are allowed

* Mentioning the default value

* Fixing comments, syncing up validations
2018-11-15 13:32:45 -06:00
debadair 19001e71cc
[DOCS] Updated release state for 7.0.0-alpha1 (#35596)
* [DOCS] Updated release state for 7.0.0-alpha1

* Update Versions.asciidoc
2018-11-15 08:22:13 -08:00
Ed Savage 2d948a001e
[HLRC][ML] Add ML delete model snapshot API (#35537)
Relates to #29827
2018-11-15 14:57:17 +00:00
Christoph Büscher bc5b1afb55
HLRC: Add parameters to stopRollupJob API (#35545)
With #34811 the API for stopping rollup jobs got two new url parameters
"wait_for_completion" and "timeout". This change adds these to the HLRC APIs as
well.

Relates to #34811
2018-11-15 10:50:48 +01:00
Clinton Gormley cb8bdeae68 Fixed bad link in ingest-geo-point 2018-11-15 09:20:04 +01:00
Tal Levy dc1821c707
explain geo_point mapping in geoip-processor (#29114)
simple docs change to add missing mapping explanation. Users may not be aware this is
a prerequisite for doing geo-queries on this enriched data.
2018-11-14 20:05:45 -08:00
Tal Levy 95a09ab2d6
[ILM] TEST: skip checking _ilm/explain response b/c flaky timing (#35562)
The docs are not resilient to timing issues where the ILM metadata is not set on newly 
created indices, so we shouldn't be so strict on the returned response
2018-11-14 16:45:06 -08:00
Deb Adair 91a38b8fe0 [DOCS] Fixed bad cross doc link. 2018-11-14 16:09:18 -08:00
debadair 38d6bb27f5
[DOCS] Added link to standalone ODBC driver doc. (#35526) 2018-11-14 15:23:28 -08:00
Tal Levy f70d4470f7
[ILM] getting started documentation (#35379) 2018-11-14 13:58:08 -08:00
Tal Levy 30a713cc3f
[ILM] add docs explaining high-level concepts (#35322)
add docs explaining high-level concepts
2018-11-14 09:23:50 -08:00
Benjamin Trent 803eccec11
HLRC: Adding ML Update Filter API (#35522)
* HLRC: Adding ml get filters api

* HLRC: Adding ML Update Filter API
2018-11-14 11:13:11 -06:00
Thiago Souza 98e075227b [Doc] Correct jdbc url example (#35513)
Adds a missing `?` to separate the url parameters.
2018-11-14 17:30:23 +01:00
Tanguy Leroux 5b7446bb5f
Add Delete Privileges API to HLRC (#35454)
This commit adds the Delete Privileges API to the high level REST
client.

Related to #29827
2018-11-14 14:04:30 +01:00
Ed Savage e7b7d52a6a
[HLRC][ML] Add ML get model snapshots API (#35487)
Relates #29827
2018-11-14 13:03:04 +00:00
Jim Ferenczi 72504c2512
Do not recommend to use the _id field in search_after docs (#35370)
The documentation of `search_after` recommends to use the `_id`
field as a tiebreaker for the sort without warning against
the additional memory required. This change changes the recommandation
to use a copy of the `_id` field with doc_values enabled.
2018-11-14 10:50:31 +01:00
Christoph Büscher 603d1a470f
Add stop rollup job support to HL REST Client (#34702)
This change adds support for stoping a rollup job to the High Level REST Client.

Relates to #29827
2018-11-14 05:40:42 +01:00
Zachary Tong c346a0f027
[Rollup] Add `wait_for_completion` option to StopRollupJob API (#34811)
This adds a `wait_for_completion` flag which allows the user to block 
the Stop API until the task has actually moved to a stopped state, 
instead of returning immediately.  If the flag is set, a `timeout` parameter
can be specified to determine how long (at max) to block the API
call.  If unspecified, the timeout is 30s.

If the timeout is exceeded before the job moves to STOPPED, a
timeout exception is thrown.  Note: this is just signifying that the API
call itself timed out.  The job will remain in STOPPING and evenutally
flip over to STOPPED in the background.

If the user asks the API to block, we move over the the generic
threadpool so that we don't hold up a networking thread.
2018-11-13 16:37:17 -05:00
Benjamin Trent b9eb5f7b63
HLRC: Adding ml get filters api (#35502)
* HLRC: Adding ml get filters api

* refactoring setId name
2018-11-13 14:53:32 -06:00
Colin Goodheart-Smithe c934fb087a
Adds HLRC docs for put lifecycle policy (#35457)
* Adds HLRC docs for put lifecycle policy

* Adds link to docs in client javadocs

* Fixes checkstyle

* Make the documentation use the right ack response
2018-11-13 20:40:53 +00:00
Lee Hinman 6cb2aef943
[ILM] Add documentation for error handling in ILM (#35426)
* [ILM] Add documentation for error handling in ILM

This adds some initial documentation for error handling and retrying failed
steps for index lifecycle management
2018-11-13 12:19:22 -07:00
lcawl 2ac4ef7a69 [DOCS] Fixes broken link 2018-11-13 10:04:26 -08:00
Jason Tedor 8b2f22ee07
Introduce CCR overview (#35436)
This commit introduces a basic overview for cross-cluster replication to
the docs.

Co-authored-by: "lcawl <lcawley@elastic.co>"
2018-11-13 12:15:37 -05:00
Luca Cavanna 0b7d18d733
HLRC: add support for get license basic/trial status API (#33176)
Relates to #29827
2018-11-13 15:52:59 +01:00
Igor Motov a76ac5729d
Rest HL client: Add watcher stats API (#35185)
Relates to #29827
2018-11-13 04:47:35 -10:00
Jason Tedor 4b5fbada9f
Fix reference to get follower stats API
This commit fixes a reference to the CCR get follower stats API in the
CCR getting started docs.
2018-11-13 09:14:26 -05:00
Przemyslaw Gomulka fd4cd80496
HLRC: migration api - upgrade (#34898)
Implement high level client for migration upgrade API. It should wrap
RestHighLevelClient and expose high level IndexUpgradeRequest (new),
IndexTaskResponse for submissions with wait_for_completion=false and
BulkByScrollResponse (already used) objects.

refers: #29827
2018-11-13 15:01:53 +01:00
Martijn van Groningen e81671dd45
[HLRC] Added support for CCR Put Follow API (#35409)
This change also adds documentation for the Put Follow API and
adds a CCR HLRC integration test.
2018-11-13 13:22:49 +01:00
Jason Tedor cbb16d5ffc
Remove extraneous spaces from CCR getting started
This commit removes some extraneous spaces that were committed to the
cross-cluster replication getting started docs.
2018-11-13 06:48:56 -05:00
Jason Tedor 92fef40dbf
Introduce CCR getting started guide (#35434)
This commit introduces a basic getting started guide for cross-cluster
replication to the docs.

Co-authored-by: "lcawl <lcawley@elastic.co>"
2018-11-13 06:45:00 -05:00
Lisa Cawley 9aeaceac4b
[DOCS] Clarify results_index_name description (#35463) 2018-11-12 13:08:57 -08:00
Zachary Tong 2da239fb5e
[HLRC] Add GetRollupIndexCaps API (#35102)
Also refactors the caps response tests a bit to share the same
abstract class to reduce duplication of test code
2018-11-12 15:05:03 -05:00
Jason Tedor 326b238852
Add docs for CCR stats API (#35439)
This commit adds API docs for the CCR stats endpoint.
2018-11-12 10:28:35 -05:00
Peter Dyson c8e685eb34 Clarify S3 repository storage class parameter (#35400)
Today it is unclear that the `storage_class` parameter to an S3 repository only
affects new objects and does not rewrite any existing objects. This commit
clarifies this point.
2018-11-12 12:04:33 +00:00
Jason Tedor eb8f346204
Fix beta label on CCR index page
This commit fixes the location of the beta label on the CCR index page.
2018-11-10 15:45:38 -05:00
Jason Tedor d84ff3765d
Mark CCR as beta in the docs
This commit marks CCR as beta in the docs. This beta label is applied to
all CCR pages.
2018-11-10 14:38:22 -05:00
Albert Zaharovits f2d7c94949
DOCS Add audit ignore settings to reference page (#35274)
Adds the logfile audit ignore policy settings synopsis
to the Auditing Reference page.
2018-11-09 16:58:10 +02:00
Tim Vernum b0acd709d7
DOCS: Add SecurityExtension breaking change (#35358)
We changed the way realm settings are defined, and this affects custom
realms in SecurityExtensions. This change adds those details to the
breaking changes docs.

Relates: #30241
2018-11-09 16:57:49 +11:00
Armin Braun 02b4e28534
#31608 Add S3 Setting to Force Path Type Access (#34721)
* SNAPSHOTS: Use Path Style Access in S3

* Use path style access pattern to fix #31608
* closes #31608
2018-11-09 05:07:26 +01:00
debadair b485e69a5f
ILM API ref edits (#35112)
* [DOCS] ILM API Ref edits

* [DOCS] Fixed endpoint for DELETE policy.

* [DOCS] Removed comparison to setting index.lifecycle.name to null.

* [DOCS] Fixed xrefs to explain API.
2018-11-08 15:26:27 -08:00
Lisa Cawley c88b2d7e5a
[DOCS] Cleans up monitoring configuration steps (#35111) 2018-11-08 13:53:09 -08:00
Mayya Sharipova 7e144b318a
Painless Context Doc: Add field context example (#35130)
* Painless Context Doc: Add field context example

relates to #34829
2018-11-08 14:54:20 -05:00
Jason Tedor 730ec1ddfb
Add more detailed OS name on Linux (#35352)
Today our OS information returned in node stats only returns a
high-level name of the OS (e.g., "Linux"). Yet, for some uses this is
too high-level and knowing at a finer level of granularity the
underlying OS can be useful. This commit extracts the pretty name on
Linux from /etc/os-release. This pretty name usually includes the Linux
vendor and the Linux vendor version number (e.g., Fedora 28).
2018-11-08 12:16:58 -05:00
Christoph Büscher 113af7996c
Make limit on number of expanded fields configurable (#35284)
Currently we introduced a hard limit of 1024 to the number of fields a query can
be expanded to in #26541. Instead of using a hard limit, we should make this
configurable. This change removes the hard limit check and uses the existing
`max_clause_count` setting instead.

Closes #34778
2018-11-08 17:04:40 +01:00
Zachary Tong 55c6481b6b
[Docs] Add painless context details for bucket_selector (#35162)
Adds docs for the bucket_selector context, an example
and corresponding doc test
2018-11-08 11:02:30 -05:00
Jim Ferenczi d96202a282 [DOCS] Fix missing callouts 2018-11-08 15:40:01 +01:00
Dominik Stadler d351422215 Add parent-aggregation to parent-join module (#34210)
Add `parent` aggregation, a special single bucket aggregation that joins children documents to their parent.
2018-11-08 14:13:00 +01:00
Albert Zaharovits 025a0c82e5
Remove deprecated audit settings (#35205)
Removes `.prefix` deprecated settings for the logfile
auditing and also documents it in the migrate asciidoc.
2018-11-08 14:06:47 +02:00
Russ Cam 848847d8c7 [Docs] Section header preceded by blank line (#34340) 2018-11-08 12:44:13 +01:00
Jeff Hajewski d00b23c8b1 Fixes fast vector highlighter docs per issue 24318. (#34190)
The `fvh` highlighter does not support span queries. This fix updates
the docs to add a warning stating the lack of span query support for
`fvh`.
2018-11-08 11:09:03 +01:00
Jason Tedor a6073f5130
Add docs on JNA temp directory not being noexec (#35355)
If the underlying mount point for the JNA temporary directory is mounted
noexec on Linux, then the JVM will not be able to map the native code in
as executable. This will prevent JNA from executing and will prevent
Elasticsearch from being able to execute some functions that rely on
native code (e.g., memory locking, and installing system call
filters). We do not want to get into the business of catching exceptions
and parsing messages towards this because these exception messages can
change on us. We also do not want to jump through a lot of hoops to
check the underlying mount point for noexec. Instead, we will rely on
documentation to address this problem. This commit adds to the important
system configuration section of the docs that the JNA temporary
directory is not on a mount point with the noexec mount option.
2018-11-07 22:25:37 -05:00
Jason Tedor 4f4fc3b8f8
Replicate index settings to followers (#35089)
This commit uses the index settings version so that a follower can
replicate index settings changes as needed from the leader.

Co-authored-by: Martijn van Groningen <martijn.v.groningen@gmail.com>
2018-11-07 21:20:51 -05:00
Alpar Torok 5ae03195d3
Make version field names more meaningful (#35334)
* Consolidate the name of the qualified build version

* Field name in response should not be redundant
2018-11-07 18:36:02 +02:00
Martijn van Groningen 021f80517f
[CCR] Added HLRC support for pause follow API (#35216)
* Moved `AcknowledgedResponse` to core package
* Made AcknowledgedResponse not abstract and provided a default parser,
so that in cases when the field name is not overwritten then there
is no need for a subclass.

Relates to #33824
2018-11-07 17:08:35 +01:00
Chris Cho e572a21c4b [Docs] Improve Convert Processor description (#35280)
Sometimes users are confused about whether they can use the Convert Processor
for changing an existing fields type to other types even if the existing one is already
ingested. This confusion is from the first line of description. Changing this and also
adding a some detail to the code snippet.
2018-11-07 17:01:35 +01:00
Alpar Torok 8a85b2eada
Remove build qualifier from server's Version (#35172)
With this change, `Version` no longer carries information about the qualifier,
we still need a way to show the "display version" that does have both
qualifier and snapshot. This is now stored  by the build and red from `META-INF`.
2018-11-07 14:01:05 +02:00
Julie Tibshirani ec53288fc0
Remove include_type_name from the relevant APIs. (#35192)
We've decided that the bulk, delete, get, index, update, and search APIs should not
contain this request parameter, and we will instead accept both typed and typeless calls.
2018-11-06 14:33:48 -08:00
Jay Modi 6f6b265166
HLRC: add support for the clear realm cache API (#35163)
This change adds support for clearing the cache of a realm. The realms
cache may contain a stale set of credentials or incorrect role
assignment, which can be corrected by clearing the cache of the entire
realm or just that of a specific user.

Relates #29827
2018-11-06 13:12:24 -07:00
Zachary Tong 065264d57c
[Docs] Add painless context details for bucket_script (#35142)
Fleshes out the details of the bucket_script context, adds an example
and corresponding doc test
2018-11-06 12:18:29 -05:00
Gytis Šk 3ee37b425b Docs: Add section about range query for range type (#35222)
This makes their interaction more discoverable.
2018-11-06 10:49:12 -05:00
Tal Levy a85b4f42ca
[ILM] change remove-policy-from-index http method from DELETE to POST (#35268)
The remove-ilm-from-index API was using the DELETE http method
to signify that something is being removed. Although, metadata
about ILM for the index is being deleted, no entity/resource
is being deleted during this operation. POST is more in line with
what this API is actually doing, it is modifying the metadata for
an index. As part of this change, `remove` is also appended to the path 
to be more explicit about its actions.
2018-11-06 07:46:25 -08:00
Jeff Soloshy 14c8a483d5 [Docs] Minor formatting and wording fixes (#35278) 2018-11-06 07:52:13 +01:00
Tim Vernum 7cd27d1a96 [DOCS] Fix missing float in breaking changes 2018-11-06 16:59:11 +11:00
Tim Vernum 3776de5f20
HLRC: Add InvalidateToken security API (#35114)
This change adds the Invalidate Token API
(DELETE /_xpack/security/oauth2/token) to the Elasticsearch
High Level Rest Client.

Relates: #29827
2018-11-06 15:26:12 +11:00
Tim Vernum 574ec6686e
Include realm type in Security Realm setting keys (#30241)
This moves all Realm settings to an Affix definition.
However, because different realm types define different settings
(potentially conflicting settings) this requires that the realm type
become part of the setting key.

Thus, we now need to define realm settings as:

    xpack.security.authc.realms:
      file.file1:
        order: 0

      native.native1:
        order: 1

- This is a breaking change to realm config
- This is also a breaking change to custom security realms (SecurityExtension)
2018-11-06 14:56:50 +11:00
lcawl 8280596f16 [DOCS] Fixes formatting error in auditing settings 2018-11-05 15:30:35 -08:00
Alexander Zhukov 842809ef37 Doc: Drop an extra 'a' in snapshots.asciidoc (#35251) 2018-11-05 13:31:35 -05:00
Nik Everett 59a43180a5 Reindex: Clean up docs around multi-index
We have an example in `reindex`'s docs about copying from many indices
at once. It doesn't work at the moment because we only allow a single
type per index. We didn't notice it in the docs tests because those
tests didn't copy any documents. This change:
1. Adds documents to the docs tests to fully exercise the snippet.
2. Fixes the example by moving all copied documents to the same type.
3. Moves the note about id collisions and expands on it because it is
even more likely than before.

Closes #35150
2018-11-05 12:52:11 -05:00
Jake Landis c2766b65cf
ingest: raise visibility of ingest plugin documentation (#35048)
* move the set security user processor to the main documentation
* link to plugin processors

part of #33188
2018-11-05 11:44:10 -06:00
Lee Hinman 3473217563
Remove Joda usage from ILM (#35220)
This commit removes the Joda time usage from ILM and the HLRC components of ILM.
It also fixes an issue where using the `?human=true` flag could have caused the
parser not to work. These millisecond fields now follow the standard we use
elsewhere in the code, with additional fields added iff the `human` flag is
specified.

This is a breaking change for ILM, but since ILM has not yet been released, no
compatibility shim is needed.
2018-11-05 08:17:15 -07:00
Benjamin Trent a4442dacd7
HLRC: Add ML API PUT filter (#35175) 2018-11-05 08:56:53 -06:00
Mayya Sharipova a2c22ad7ae
Small corrections to HLRC doc for _termvectors (#35221)
Relates to #33447
2018-11-04 18:29:07 -05:00
Jack Conradson 44f08717ba
[Scripting] Make Max Script Length Setting Dynamic (#35184)
This changes the current script.max_size_in_bytes to be dynamic so it can be 
set through the cluster settings API. This setting is also applied to inline scripts 
in the compile method of ScriptService to prevent excessively long inline 
scripts from being compiled. The script length limit is removed from Painless as 
this is no longer necessary with the protection in compile.
2018-11-02 16:07:54 -07:00
Lisa Cawley 3c36ba1f5e
[DOCS] Move CCR overview pages (#34918) 2018-11-02 14:23:13 -07:00
Gordon Brown 9b3d581339 [DOCS] Remove link to nonexistent ILM API 2018-11-02 09:38:40 -06:00
Milan Mrdjen 34677b9c83 HLRC: Add document _count API (#34267)
Add `count()` api method, `CountRequest` and `CountResponse` classes to HLRC. Code in server module is unchanged.

Relates to #27205
2018-11-02 08:21:19 -05:00
Colin Goodheart-Smithe 8f592bda27
Removes link to deleted page 2018-11-02 12:27:51 +00:00
Daniel Mitterdorfer ccbe80c3a0
Introduce durability of circuit breaking exception
With this commit we differentiate between permanent circuit breaking
exceptions (which require intervention from an operator and should not
be automatically retried) and transient ones (which may heal themselves
eventually and should be retried). Furthermore, the parent circuit
breaker will categorize a circuit breaking exception as either transient
or permanent based on the categorization of memory usage of its child
circuit breakers.

Closes #31986
Relates #34460
2018-11-02 13:12:44 +01:00
Colin Goodheart-Smithe fc6e1f7f3f
Merge branch 'master' into index-lifecycle 2018-11-02 10:56:35 +00:00
lcawl f97c9b6af2 [DOCS] Fixes location of beta markup 2018-11-01 14:27:49 -07:00
Julie Tibshirani fda173d7aa
Add a note around using separate indices for percolator queries and documents. (#35109) 2018-11-01 12:41:07 -07:00
lcawl 781130c7ab [DOCS] Fixes formatting of dynamic settings links 2018-11-01 11:52:41 -07:00
Tal Levy c6c01425bb Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-11-01 11:38:42 -07:00
Tal Levy c3cf7dd305 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-11-01 10:13:02 -07:00
Jake Landis 77fab62ebe
ingest: add common options to each processor's documentation (#35091)
* adds `if`, `on_failure`, `tag`, and `ignore_failure` to table for each processor

part of #33188

* added ingore_failure

* fix whitespace noise
2018-11-01 11:08:04 -05:00
Itamar Syn-Hershko d46f3528c0 [Docs] Updating URL for Openstack Swift plugin (#34136)
The repository plugin for Openstack Swift was developed originally by Wikimedia foundation but is now retired. 
This changes the link to the repo where the actively maintained fork lives now.
2018-11-01 11:57:55 +01:00
Ke Li 14f540e8e6 Deprecate unicodeSetFilter in favour of unicode_set_filter (#29215) 2018-11-01 10:06:51 +00:00
Alexandru Rusanescu f3e150b0ea [Docs] Update query_cache.asciidoc (#33340)
Add note about non-visibility of cache content.
2018-11-01 10:22:36 +01:00
lipsill d181d1bab1 Remove deprecated url parameters `_source_include` and `_source_exclude` (#35097)
Removes `_source_include` and `_source_exclude` url parameters. 
These parameters have been deprecated in #33475.

Closes #22792
2018-10-31 17:11:59 -04:00
Qian Hong f27251dfcb [Docs] Improve Redindex/Update/Delete By Query. (#35127)
Fix a copy paste error and improve formatting in
Redindex/Update/Delete By Query and fix formula in Update By Query.
2018-10-31 14:04:48 -04:00
Tal Levy d5d28420b6 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-31 10:47:07 -07:00
Luca Cavanna ef5181c678
Allow to enable pings for specific remote clusters (#34753)
When we connect to remote clusters, there may be a few more routers/firewalls in-between compared to when we connect to nodes in the same cluster. We've experienced cases where firewalls drop connections completely and keep-alives seem not to be enough, or they are not properly configured. With this commit we allow to enable application-level pings specifically from CCS nodes to the selected remote nodes through the new setting `cluster.remote.${clusterAlias}.transport.ping_schedule`.  The new setting is similar `transport.ping_schedule` but it does not affect intra-cluster communication, pings are only sent to specific remote cluster when specifically enabled, as they are disabled by default.

Relates to #34405
2018-10-31 17:32:53 +01:00
Lisa Cawley 72cb885abc
[DOCS] Updates monitoring diagram (#35096) 2018-10-31 09:18:14 -07:00
Mayya Sharipova 0b8465f2c0
Update type to "seat" (#35086)
otherwise this throws an exception: java.lang.IllegalArgumentException: Rejecting mapping update to [seats] as the final mapping would have more than 1 type: [seat, _doc]

May be, later for the types removal, we can modify `seats.json` to have a type `_doc` instead of `seat`
2018-10-31 12:06:48 -04:00
Sue Gallagher 1ce3c92a2d [DOCS] Add info on calendar vs fixed interval. (#31638)
Extensive edit to add additional information on the difference between calendar intervals and fixed-length intervals.
2018-10-31 10:16:36 -04:00
Albert Zaharovits bdd8460906
HLRest: add security authenticate API (#33552)
This adds the security `_authenticate` API to the HLREST client.
It is unlike some of the other APIs because the request does not
have a body.
The commit also creates the `User` entity. It is important
to note that the `User` entity does not have the `enabled`
flag. The `enabled` flag is part of the response, alongside
the `User` entity.
Moreover this adds the `SecurityIT` test class 
(extending `ESRestHighLevelClientTestCase`).

Relates #29827
2018-10-31 14:49:33 +02:00
debadair e540abd551
[DOCS] Fixed edit links for ILM APIs and added the APIs to the REST ref (#35070)
* [DOCS] Fixed edit links for ILM APIs and added the APIs to the REST API section.

* [DOCS] Fixed link to ILM APIs.
2018-10-30 18:27:43 -07:00
Tal Levy 5141084048
rename CRUD api REST path prefix _ilm to _ilm/policy (#35056)
This PR renames the CRUD APIS for ILM

GET _ilm/<policy>, _ilm -> _ilm/policy/<policy>, _ilm/policy
PUT _ilm/<policy> -> _ilm/policy/<policy>
DELETE _ilm/<policy> -> _ilm/policy/<policy>

closes #34929.
2018-10-30 16:19:05 -07:00
Tal Levy f5323cd86e
rename ILM Phase "minimum_age" to "min_age" (#35058)
closes #34927.
2018-10-30 12:54:02 -07:00
Nik Everett 0f9a7b3344 Docs: Fix build after moving SQL QA
I moved SQL's QA in #34496 to make it easier to run all of SQL tests in
one go but forgot to update the docs build. This fixes that.
2018-10-30 15:47:25 -04:00
Nik Everett 2b2e208c4c
Docs: Remove range notation from random score docs (#35093)
The `random_score` function produces values between 0 (inclusive) and 1
(exclusive) and documented it with fancy methematical range notation. It
is so fancy I thought it was a typo. This changes the documentation to
use words.

Relates to #35084
2018-10-30 14:12:59 -04:00
Zachary Tong f9dd33a0b9
[Rollup] Proactively resolve index patterns in RollupSearch endoint (#34930)
This changes the RollupSearch endpoint to proactively resolve index
patterns.  If the index pattern(s) match more than one rollup index,
an exception is throw as before.  But if the pattern only matches one
rollup index, execution is allowed to continue (unlike before where
it would assume all patterns were for raw data).

This also allows the search endpoint to resolve aliases that point to
a rollup index.

Also tweaks the documentation to make this clear.

Closes #34828
2018-10-30 13:50:50 -04:00
Julie Tibshirani 70da490f34
Remove some documentation that only makes sense with multiple types. (#35066)
* Remove a tip about ignore_above that only makes sense with multiple types.
* Remove a line from the percolator documentation that refers to multiple types.
2018-10-30 10:19:12 -07:00
Tal Levy 18c72e86c5 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-30 08:09:57 -07:00
Andy Bristol b8280ea7cc
median absolute deviation agg (#34482)
This commit adds a new single value metric aggregation that calculates
the statistic called median absolute deviation, which is a measure of
variability that works on more types of data than standard deviation

Our calculation of MAD is approximated using t-digests. In the collect
phase, we collect each value visited into a t-digest. In the reduce
phase, we merge all value t-digests, then create a t-digest of
deviations using the first t-digest's median and centroids
2018-10-30 07:22:52 -07:00
Nik Everett cc4dd98872 Docs: Fix build for java-api
The java-api docs had some broken references. Mostly because it is
difficult to build them so they are difficult to check.

Relates to #34528
2018-10-30 09:14:07 -04:00
Alan Woodward cf24231b1b
Add missing full stop in Bootstrap checks doc (#35078)
Fixes #35075
2018-10-30 11:43:43 +00:00
markwalkom 5e0c9b7c1a Update reindex.asciidoc (#35062)
Highlighted that `_source` needs to exist in the source index.
2018-10-30 10:25:05 +01:00
Przemyslaw Gomulka 995bf0ee66
Bulk Api support for global parameters (#34528)
Bulk Request in High level rest client should be consistent with what is
possible in Rest API, therefore should support global parameters. Global
parameters are passed in URL in Rest API.

Some parameters are mandatory - index, type - and would fail validation
if not provided before before the bulk is executed.
Optional parameters - routing, pipeline.

The usage of these should be consistent across sync/async execution,
bulk processor and BulkRequestBuilder

closes #26026
2018-10-30 09:08:12 +01:00
Ken MacInnis d67c88fa12 [Docs] Update update-settings.asciidoc (#35073)
Add explanation for ?include_defaults parameter and link to
Cluster Get Settings reference
2018-10-30 08:58:32 +01:00
Gordon Brown 794d4fa879
Label required scripts in Scripted Metric Agg docs (#35051)
When combine_script and reduce_script were made into required
parameters for Scripted Metric aggregations in #33452, the docs were
not updated to reflect that. This marks those parameters as required
in the documentation.
2018-10-29 15:13:14 -06:00
Tal Levy c9e4d26a53 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-29 14:03:55 -07:00
Nik Everett 3051e37e45 Docs: Document refresh for reindex and friends
Reindex and friends don't support `refresh=wait_for` and this documents
that.
2018-10-29 13:58:55 -04:00
lipsill 6df1c9e818 Deprecate `_source_include` and `_source_exclude` url parameters (#33475)
Deprecates `_source_include` and `_source_exclude` url parameters
in favor of `_source_inclues` and `_source_excludes` because those
are consistent with the rest of Elasticsearch's APIs.

Relates to #22792
2018-10-29 12:06:38 -04:00
Nik Everett 6fd7ea71f1
HLRC: DRY up remaining CRUD docs (#34925)
This further applies the pattern set in #34125 to reduce copy-and-paste
in the multi-document CRUD portion of the High Level REST Client docs.
It also adds line wraps to snippets that are too wide to fit into the box
when rendered in the docs, following up on the work started in #34163.
2018-10-29 10:32:17 -04:00
Jason Tedor 26f5c509af
Fix CCR API specification (#34963)
This commit fixes two issues with the CCR API specification:
 - remove the CCR stats endpoint, it is not currently implemented
 - fix the documentation links
2018-10-29 09:37:13 -04:00
Christoph Büscher 389910f11d
[Docs] Add migration note about expanded fields limit (#34920)
Adds a note to warn users about the limit introduced in #26541.
2018-10-29 10:23:07 +01:00
David Roberts c455be7bc2
[ML] Rename the json file structure to ndjson (#34901)
The file structure finder endpoint can find the NDJSON
(newline-delimited JSON) file format, but called it
`json`.  This change renames the `format` for this file
structure to `ndjson`, which is more precise and will
hopefully avoid confusion.
2018-10-29 10:06:12 +01:00
Martijn van Groningen f13d529448
Fixed ccr docs broken links. 2018-10-29 09:29:28 +01:00
Christoph Büscher 3a464bdee8
Add start rollup job support to HL REST Client (#34623)
This change adds support for starting a rollup job to High Level REST Client.

Relates to #29827
2018-10-29 09:28:11 +01:00
Martijn van Groningen 1801518527
[CCR] Refactor stats APIs (#34912)
* Changed the auto follow stats to also include follow stats.
* Renamed the auto follow stats api to stats api and changed its url path
  from `/_ccr/auto_follow/stats` `/_ccr/stats`.
* Removed `/_ccr/stats` url path for the follow stats api, which makes
  the index parameter a required parameter.
* Fixed docs.
2018-10-29 07:45:27 +01:00
Tim Vernum 9c27b407f0
HLRC: Add security Create Token API (#34791)
This adds the Create Token API (POST /_xpack/security/oauth2/token)
to the High Level Rest Client.

Relates: #29827
2018-10-29 17:17:56 +11:00
Yogesh Gaikwad dc5bfe3a00
[DOCS] Fix typo in get role mappings API (#34952) 2018-10-29 11:15:42 +11:00
Yogesh Gaikwad a5ee134c40
[HLRC] Add support for get role mappings API (#34637)
This commit adds support for get role mappings API
in HLRC.
2018-10-29 10:12:13 +11:00
debadair 544c220942
[DOCS] Added attributes for HLRC javadoc packages. (#34934) 2018-10-26 15:32:15 -07:00
Benjamin Trent 052dfa5646
HLRC: Adding Update datafeed API (#34882)
* HLRC: Adding Update datafeed API

* Addressing unused import

* Adjusting docs and fixing minor comments

* fixing comment
2018-10-26 16:44:12 -05:00
Tal Levy d8322ca069 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-26 12:46:21 -07:00
Tal Levy e737ea7d4a
remove old doc placeholder and migrate ilm docs to top-level (#34615)
we are restructuring the docs, this migrates ILM docs outside of the x-pack doc structure.
2018-10-26 12:19:52 -07:00
Boaz Leskes a086c665a3 HLREST: Add Clear Roles Cache API (#34187)
Adds support for the Clear Roles Cache API to the High Level Rest
Client. As part of this a helper class, NodesResponseHeader, has been
added that enables parsing the nodes header from responses that are
node requests.

Relates to #29827
2018-10-26 12:16:44 -06:00
Jason Tedor fdfdbe486d
Introduce cross-cluster replication API docs (#34726)
This commit is our first introduction to cross-cluster replication
docs. In this commit, we introduce the cross-cluster replication API
docs. We also add skelton docs for additional content that will be added
in a series of follow-up commits.
2018-10-26 11:23:35 -04:00
markharwood 3181083781
HLRC - add support for source exists API (#34519)
HLRC - add support for source exists API
API re-uses the GetRequest object (following the precedent set by the plain “exists” api).

Relates to #27205
2018-10-26 14:21:35 +01:00
Albert Zaharovits 3f1fec1813
[Docs] audit logfile structured format (#34584)
Documents the new structured logfile format for auditing
that was introduced by #31931. Most changes herein
are for 6.x . In 7.0 the deprecated format is gone and a
follow-up PR is in order.
2018-10-26 15:19:35 +03:00
Tal Levy 810cd46a30 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-25 14:35:33 -07:00
Senthil e27a0d5b0d Docs: Align prose with snippet (#34839)
Modified example text to include multiple index patterns used in the code
2018-10-25 16:51:18 -04:00
Stéphane Campinas 27c4d63340 document the search context is freed if the scroll is not extended (#34739)
The `fetchPhaseShouldFreeContext` returns true when there is a scroll context but the scroll parameter is null, thus freeing the search context.

183c32d4c3/server/src/main/java/org/elasticsearch/search/SearchService.java (L491)
2018-10-25 16:49:08 -04:00
Lisa Cawley 312df5546c
[DOCS] Updates Elasticsearch monitoring tasks (#34339) 2018-10-25 10:32:50 -07:00
Lee Hinman 3e7042832a Merge remote-tracking branch 'origin/master' into index-lifecycle 2018-10-25 11:00:36 -06:00
Christoph Büscher c0c6a28e86
[Docs] Add `indices.query.bool.max_clause_count` setting (#34779)
This change adds a section about the global search setting
`indices.query.bool.max_clause_count` that limits the number of boolean clauses
allowed in a Lucene BooleanQuery.

Closes #19858
2018-10-25 17:59:59 +02:00
Jason Tedor 153157e56d
Enable ingest attachment docs tests on JDK 11 (#34770)
These were disabled because ingest attachement was not playing well with
JDK 11. We have addressed that by upgrading the Tika dependency, yet
forgot to reenable these. This commit enables these tests again.
2018-10-24 23:17:45 -04:00
Mayya Sharipova d7afd7c123 Reduce the number of callouts to 15 in docs
relates to #33447
2018-10-24 17:57:35 -04:00
Mark Tozzi d94406a68a
HLRC: Deactivate Watch API (#34192)
Relates to #29827
2018-10-24 17:01:22 -04:00
Mayya Sharipova 3211760e2c Correct term-vectors.asciidoc
relates to #33447
2018-10-24 16:35:20 -04:00
Benjamin Trent cd27b0b996
Revert "Rollup add default metrics to histo groups (#34534)" (#34815)
This reverts commit 4236358f5d.
2018-10-24 14:25:10 -05:00
Mayya Sharipova bf4d90a5dc
HLRC API for _termvectors (#33447)
* HLRC API for _termvectors

relates to #27205
2018-10-24 14:27:22 -04:00
Gordon Brown da20dfd81c
Add cluster-wide shard limit warnings (#34021)
In a future major version, we will be introducing a soft limit on the
number of shards in a cluster based on the number of nodes in the
cluster. This limit will be configurable, and checked on operations
which create or open shards and issue a warning if the operation would
take the cluster over the limit.

There is an option to enable strict enforcement of the limit, which
turns the warnings into errors.  In a future release, the option will be
removed and strict enforcement will be the default (and only) behavior.
2018-10-23 16:35:10 -06:00
Armin Braun f79bdec58a INGEST: Document Pipeline Processor (#33418)
* Added documentation for Pipeline Processor
* Relates #33188
2018-10-23 15:36:57 -05:00
Zachary Tong 299d044bfc
Collapse pipeline aggs into single package (#34658)
- Restrict visibility of Aggregators and Factories
- Move PipelineAggregatorBuilders up a level so it is consistent with
AggregatorBuilders
- Checkstyle line length fixes for a few classes
- Minor odds/ends (swapping to method references, formatting, etc)
2018-10-23 16:01:01 -04:00
Jake Landis 067d135bfd
ingest: document index.default_pipeline (#34500) 2018-10-23 13:49:25 -05:00
Jake Landis a8e1ee34ca
ingest: document fields that support templating (#34536)
This change also updates many of the examples to use ecs as the example.
Some additional minor improvements are also included.

Part of #33188
2018-10-23 13:28:44 -05:00
Jake Landis c447fc258a
ingest: documentation for the drop processor (#34570) 2018-10-23 12:30:23 -05:00
Tal Levy 62ac2fa5ec Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-23 09:43:46 -07:00
Zachary Tong 4dbf498721
[Rollup] Job deletion should be invoked on the allocated task (#34574)
We should delete a job by directly talking to the allocated 
task and telling it to shutdown. Today we shut down a job 
via the persistent task framework. This is not ideal because, 
while the job has been removed from the persistent task 
CS, the allocated task continues to live until it gets the 
shutdown message.

This means a user can delete a job, immediately delete 
the rollup index, and then see new documents appear in
 the just-deleted index. This happens because the indexer
 in the allocated task is still running and indexes a few 
more documents before getting the shutdown command.

In this PR, the transport action is changed to a TransportTasksAction, 
and we invoke onCancelled() directly on the matching job. 
The race condition still exists after this PR (albeit less likely), 
but this was a precursor to fixing the issue and a self-contained
chunk of code. A second PR will followup to fix the race itself.
2018-10-23 12:23:22 -04:00
Albert Zaharovits 46b49b01c4
[DOCS] .Security index is never auto created (#34589) 2018-10-23 18:51:59 +03:00
Armin Braun f0f732908e
INGEST: Document Processor Conditional (#33388)
* INGEST: Document Processor Conditional

Relates #33188
2018-10-23 17:37:30 +02:00
Costin Leau ca6808e55d
SQL: Support pattern against compatible indices (#34718)
Extend querying support on multiple indices from being strictly
identical to being just compatible.
Use FieldCapabilities API (extended through #33803) for mapping merging.

Close #31837 #31611
2018-10-23 17:07:51 +03:00
Przemyslaw Gomulka f91cc13c18
A small typo in migration-assistance doc (#34704)
typo - missing word "be"
2018-10-23 15:28:25 +02:00
Marios Trivyzas 4a8386f271
SQL: Implement IN(value1, value2, ...) expression. (#34581)
Implement the functionality to translate the
`field IN (value1, value2,...)` expressions to proper Lucene queries
or painless script or local processors depending on the use case.

The `IN` expression can be used in SELECT, WHERE and HAVING clauses.

Closes: #32955
2018-10-23 14:28:23 +02:00
Frederik Bosch 183c32d4c3 [DOCS] Force Merge: clarify execution and storage requirements (#33882) 2018-10-23 12:25:04 +02:00
Marios Trivyzas e9e140790a
SQL: Implement `CONVERT`, an alternative to `CAST` (#34660)
`CONVERT` works exactly like cast with slightly different syntax:
`CONVERT(<value>, <data_type)` as opposed to `CAST(<value> AS <data_type>)`

Moreover it support format of the MS-SQL data types `SQL_<type>`,
e.g.: `SQL_INTEGER`

Closes: #34513
2018-10-23 11:21:15 +02:00
Tal Levy 67bfdb16ad Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-22 13:09:37 -07:00
Julie Tibshirani f854330e06
Make sure to use the type _doc in the REST documentation. (#34662)
* Replace custom type names with _doc in REST examples.
* Avoid using two mapping types in the percolator docs.
* Rename doc -> _doc in the main repository README.
* Also replace some custom type names in the HLRC docs.
2018-10-22 11:54:04 -07:00
Zachary Tong d981746142
[Docs] clarification about cardinality accuracy (#34616)
Adds a bit more clarification about how accuracy is dependent
on the dataset in question.

Closes #18231
2018-10-22 13:15:45 -04:00
Julie Tibshirani fbb9ac34f9
Deprecate type exists requests. (#34663) 2018-10-22 08:46:11 -07:00
Christoph Büscher e869f9a78c [Docs] Update synonym-tokenfilter.asciidoc (#34706)
Remove ugly double-dot.
2018-10-22 17:18:29 +02:00
Jason Tedor bf5f0af491
Use trial license in docs tests (#34673)
This commit switches to using a trial license in the docs tests that run
on the default distribution. This is needed so that docs tests can be
executed against non-basic features.
2018-10-21 06:23:42 -04:00
Ryan Ernst 222652dfce
Scripting: Convert script fields to use script context (#34164)
This commit removes the use of SearchScript for script fields and adds
a new FieldScript.
2018-10-20 16:33:49 -07:00
Jason Tedor aebb855f38
Add missing word to remote clusters docs
This commit adds a missing word in the initial section of the remote
clusters docs.
2018-10-20 12:11:41 -04:00
Jason Tedor aca8c705ae
Fix name of remote clusters reference
This commit fixes the name of the reference to the remote clusters
module on the modules page by adding a missing character.
2018-10-20 11:18:27 -04:00
Jason Tedor ab5a0b15f6
Fix formatting of remotes clusters module section
This commit fixes the formatting of the remote clusters module section
on the modules page. We add a missing newline and wrap the columns at
80.
2018-10-20 09:04:48 -04:00
Jason Tedor 91dbb22950
Fix reference to remote clusters docs
This commit fixes a reference to the remote clusters docs from the
modules page.
2018-10-20 09:02:00 -04:00
Jason Tedor b2dc58450e
Separate remote clusters docs from CCS (#34612)
With remote clusters taking on a larger role, we have make the
infrastructure more generic than being tied to cross-cluster search
(CCS). We want to refer to the remote clusters configuration in the
cross-cluster replication (CCR) docs. Yet, these docs are still tied to
CCS. This commit extracts the remote clusters docs from CCS (with some
wording changes to make them more general) so that we can refer to them
in the CCR docs.
2018-10-20 08:33:59 -04:00
Yannick Welsch 9fb2f7cc20
HLRC: Delete role API (#34620)
Adds the "Delete role" API to the high-level REST client.
2018-10-20 12:11:36 +02:00
Jack Conradson 842bf55497 [Docs] Fixes a bug in the rest-client watcher docs related to
cross-link issue.
2018-10-19 11:52:15 -07:00
Igor Motov 94bde37bcf
Geo: Don't flip longitude of envelopes crossing dateline (#34535)
When a envelope that crosses the dateline is specified as a part of
geo_shape query is parsed it shouldn't have its left and right points
flipped.

Fixes #34418
2018-10-19 13:53:54 -04:00
Julie Tibshirani 67652b5355
Remove references to multiple types in the search documentation. (#34625) 2018-10-19 09:47:34 -07:00
Jack Conradson 1100942455
[HLRC] Add Start/Stop Watch Service APIs. (#34317) 2018-10-19 09:42:57 -07:00
Costin Leau 52104aac27
SQL: Introduce support for NULL values (#34573)
Make SQL aware of missing and/or unmapped fields treating them as NULL
Make _all_ functions and operators null-safe aware, including when used
in filtering or sorting contexts
Add missing and null-safe doc value extractor
Modify dataset to have null fields spread around (in groups of 10)
Enforce missing last and unmapped_type inside sorting
Consolidate Predicate templating and declaration
Add support for Like/RLike in scripting
Generalize NULLS LAST/FIRST
Introduce early schema declaration for CSV spec tests: to keep the doc
snippets in place (introduce schema:: prefix for declaration)
upfront.

Fix #32079
2018-10-19 16:44:33 +03:00
Colin Goodheart-Smithe 84ef91529c
Merge branch 'master' into index-lifecycle 2018-10-19 13:24:04 +01:00
Benjamin Trent 4236358f5d
Rollup add default metrics to histo groups (#34534)
* Rollup: Adding default metrics for histo group timefield (#34379)

* Rollup: Adding default histo metrics and tests

* fixing failing client side test with new default values

* Adding HLRC docs for default values

* Addressing PR comments

* Removing value_count default agg

* Updating docs for rollups

* Minor interval change
2018-10-19 07:23:25 -05:00
Jim Ferenczi 7b49beb9b0
Fix threshold frequency computation in Suggesters (#34312)
The `term` and `phrase` suggesters have different options to filter candidates
based on their frequencies. The `popular` mode for instance filters candidate
terms that occur in less docs than the original term. However when we compute this threshold
we use the total term frequency of a term instead of the document frequency. This is not inline
with the actual filtering which is always based on the document frequency. This change fixes
this discrepancy and clarifies the meaning of the different frequencies in use in the suggesters.
It also ensures that the threshold doesn't overflow the maximum allowed value (Integer.MAX_VALUE).

Closes #34282
2018-10-19 13:33:19 +02:00
markharwood fe623acf66
Docs - removed experimental/beta markers from adjacency matrix aggregation (#34599) 2018-10-19 09:33:59 +01:00
markharwood 2a413abb0b
Docs - remove experimental marker from significant_text aggregation (#34598) 2018-10-19 09:32:02 +01:00
Yogesh Gaikwad 39a6163316
[HLRC] Add support for Delete role mapping API (#34531)
Building on expression dsl and create role mapping API, this
commit adds support for delete role mapping API to high level client.
2018-10-19 13:46:26 +11:00
Christoph Büscher 7cc3ac0fc4
[Docs] Add _cache/clear parameter examples (#34614)
Add example for selectively clearing just the request, query or fielddata cache
and for selectively clearing the cache for specific fields.

Closes #34287
2018-10-19 00:44:37 +02:00
Zachary Tong 45546e71c2
Add GetRollupCaps API to high level rest client (#32880)
Adds GetRollupCaps API to the HLRC, and tweaks some of the
Caps objects to be immutable.  Also various style tweaks
2018-10-18 17:12:38 -04:00
Benjamin Trent fa973007b8
[DOCS] Adds new lazy ml node setting (#34600)
* Adding new xpack.ml.max_lazy_ml_nodes setting to docs

* Fixing docs, making it clearer what the setting does

* Adding note about external process need
2018-10-18 16:11:36 -05:00
Nik Everett 2c0f67318a
Docs: Reenable rollup docs tests (#34564)
We'd disabled them because we didn't have a way to clean up after each
test. I implemented #34342 which adds the clean ups so now we can
re-enable the tests.

In the `setup` sections we have to use `raw` requests instead of
`x-pack` requests because we don't have the json config for x-pack.

Closes #33319
2018-10-18 15:24:02 -04:00
Ryan Ernst d445785f1a
Scripting: Convert domainSplit function for ML to whitelist (#34426)
This commit moves the definition of domainSplit into java and exposes it
as a painless whitelist extension. The method also no longer needs
params, and version which ignores params is added and deprecated.
2018-10-17 15:54:21 -07:00
Colin Goodheart-Smithe 90f7cec7a5
Merge branch 'master' into index-lifecycle 2018-10-17 18:22:23 +01:00
Andy Bristol 18aa1c1381
add start trial API to HLRC (#33406)
Introduces client-specific request and response classes that do not
depend on the server

The `type` parameter is named `licenseType` in the response class to be
more descriptive. The parts that make up the acknowledged-required
response are given slightly different names than their server-response
types to be consistent with the naming in the put license API

Tests do not cover all cases because the integ test cluster starts up
with a trial license - this will be addressed in a future commit
2018-10-17 08:02:04 -07:00
Nik Everett 030e8c8fe3 Docs: Tweak upgrade docs
Tweak the upgrade instructions for moving from pre-6.3-with-x-pack to
post-6.3-default distribution. Specifically, you have to remove the
x-pack plugin before upgrading because 6.4 doesn't understand how to
remove it.

Relates to #34307
2018-10-17 08:17:12 -04:00
Tal Levy fbe8dc014c Merge branch 'master' into index-lifecycle 2018-10-16 13:58:53 -07:00
lipsill 9b5eaafc24 HLRC: Documentation examples cleanup (#34009)
* Replace deprecated field `code` with `source` for stored scripts (#25127)
* Replace examples using the deprecated endpoint `{index}/{type}/_search`
with `{index}/_search` (#29468)
* Use a system property to avoid deprecation warnings after the Update
Scripts have been moved to their own context (#32096)
2018-10-16 10:46:14 -04:00
Vladimir Dolzhenko 230ad5339b
HLRC: Add support for XPack Post Start Basic Licence API (#33606)
Relates to #29827
2018-10-16 15:21:30 +02:00
Paul Sanwald 936faba6c4
Add delete rollup job support to HL REST Client (#34066)
Add support for delete rollup job to HL REST Client.
2018-10-16 09:02:29 -04:00
Jim Ferenczi 544de13d8e
Disallow negative query boost (#34486)
This change disallows negative query boosts. Negative scores are not allowed in Lucene 8 so
it is easier to just disallow negative boosts entirely. We should also deprecate negative boosts
in 6x in order to ensure that users are aware when they'll upgrade to ES 7.

Relates #33309
2018-10-16 11:31:53 +01:00
Yogesh Gaikwad a4c302067e
HLRC: Create/Update role mapping API (#34171)
We added support for role mapper expression DSL in #33745,
that allows us to build the role mapper expression used in the
role mapping (as rules for determining user roles based on what
the boolean expression resolves to).

This change now adds support for create/update role mapping
API to the high-level rest client.
2018-10-16 03:05:46 +01:00
Nikolay Vasiliev f5641e61a2 Docs: improve formatting of Query String Query doc page (#34432)
Merge two tables.
2018-10-15 15:30:48 -04:00
Ioannis Kakavas 55eaf7a3ff
HLRC: Get SSL Certificates API (#34135)
This change adds support for the get SSL certificate API to 
the high level rest client.
2018-10-15 17:20:34 +01:00
Colin Goodheart-Smithe 0b42eda0e3
Merge branch 'master' into index-lifecycle 2018-10-15 16:03:37 +01:00
David Roberts 21c759af0e
[ML] Add an ingest pipeline definition to structure finder (#34350)
The ingest pipeline that is produced is very simple.  It
contains a grok processor if the format is semi-structured
text, a date processor if the format contains a timestamp,
and a remove processor if required to remove the interim
timestamp field parsed out of semi-structured text.

Eventually the UI should offer the option to customize the
pipeline with additional processors to perform other data
preparation steps before ingesting data to an index.
2018-10-12 07:56:35 +01:00
Andrei Stefan d7a94fb6aa
SQL: Functions enhancements (OCTET_LENGTH function, order functions alphabetically, RANDOM function docs) (#34101)
* New OCTET_LENGTH function
* Changed the way the FunctionRegistry stores functions, considering the alphabetic ordering by name
* Added documentation for the RANDOM function
2018-10-09 00:20:18 +03:00
Jim Ferenczi 36557469f6
[DOCS] Removes beta label from composite aggregation (#34329) 2018-10-05 19:46:20 +02:00
Jim Ferenczi 6e28c8f1c4
[DOCS] Remove experimental label from term_set query (#34328) 2018-10-05 19:45:23 +02:00
Jim Ferenczi cfe8eab455
[DOCS] Removes beta label from index sorting (#34327) 2018-10-05 19:44:25 +02:00
Benjamin Trent 6f5daadb3c
Feature/hlrc ml docs cleanup (#34316)
* HLRC: ML Add preview datafeed api

* Changing deprecation handling for parser

* Removing some duplication in docs, will address other APIs in another PR

* HLRC: ML Cleanup docs

* updating get datafeed stats docs
2018-10-05 08:45:46 -07:00
Nik Everett a9fa5f2b33
Docs: DRY up CRUD docs (#34203)
This further applies the pattern set in #34125 to reduce copy-and-paste
in the single document CRUD portion of the High Level REST Client docs.
It also adds line wraps to snippets that are too wide to fit into the box
when rendered in the docs, following up on the work started in #34163.
2018-10-05 11:41:03 -04:00
amoreauCoveo e95dc5474f Minor corrections in geo-queries.asciidoc (#34314) 2018-10-05 17:12:18 +02:00
Jim Ferenczi a9daa5cb90
[DOCS] Remove beta label from normalizers (#34326) 2018-10-05 15:42:00 +02:00
eray daf88335d7 Add max_children limit to nested sort (#33587)
Add an option to `nested` sort to limit the number of children to visit when picking the sort value
of the root document. 

Closes #33592
2018-10-05 12:02:47 +02:00
Ioannis Kakavas f420eebc73 [DOCS] Fix typo and add [float] 2018-10-05 10:07:50 +03:00
Tim Vernum 63dbd1dce0
Allow User/Password realms to disable authc (#34033)
The "lookupUser" method on a realm facilitates the "run-as" and
"authorization_realms" features.
This commit allows a realm to be used for "lookup only", in which
case the "authenticate" method (and associated token methods) are
disabled.
It does this through the introduction of a new
"authentication.enabled" setting, which defaults to true.
2018-10-05 12:10:42 +10:00
Tim Vernum 6608992523
Enable security automaton caching (#34028)
Building automatons can be costly. For the most part we cache things
that use automatons so the cost is limited.
However:
- We don't (currently) do that everywhere (e.g. we don't cache role
  mappings)
- It is sometimes necessary to clear some of those caches which can
  cause significant CPU overhead and processing delays.

This commit introduces a new cache in the Automatons class to avoid
unnecesarily recomputing automatons.
2018-10-05 12:09:36 +10:00
Dimitris Athanasiou 4dacfa95d2
[ML] Allow asynchronous job deletion (#34058)
This changes the delete job API by adding
the choice to delete a job asynchronously.
The commit adds a `wait_for_completion` parameter
to the delete job request. When set to `false`,
the action returns immediately and the response
contains the task id.

This also changes the handling of subsequent
delete requests for a job that is already being
deleted. It now uses the task framework to check
if the job is being deleted instead of the cluster
state. This is a beneficial for it is going to also
be working once the job configs are moved out of the
cluster state and into an index. Also, force delete
requests that are waiting for the job to be deleted
will not proceed with the deletion if the first task
fails. This will prevent overloading the cluster. Instead,
the failure is communicated better via notifications
so that the user may retry.

Finally, this makes the `deleting` property of the job
visible (also it was renamed from `deleted`). This allows
a client to render a deleting job differently.

Closes #32836
2018-10-05 02:41:28 +03:00
Benjamin Trent 026488bcbf
HLRC: ML Adding get datafeed stats API (#34271)
* HLRC: ML Adding get datafeed stats API

* addressing PR comments

* fixing field exclusion filter

* removing unnecessary whitespace
2018-10-04 14:43:06 -07:00
Julie Tibshirani a6c5b6807f
Small fixes to the HLRC watcher documentation. (#34306)
* Make sure to use the new docs style for 'ack watch'.
* Replace 'document' with 'watcher' in page names.
2018-10-04 23:30:20 +02:00
Nik Everett 09aaed4fe4
Tasks: Document that status is not semvered (#34270)
The `status` part of the tasks API reflects the internal status of a
running task. In general, we do not make backwards breaking changes to
the `status` but because it is internal we reserve the right to do so. I
suspect we will very rarely excercise that right but it is important
that we have it so we're not boxed into any particular implementation
for a request.

In some sense this is policy making by documentation change. In another
it is clarification of the way we've always thought of this field.

I also reflect the documentation change into the Javadoc in a few
places. There I acknowledge Kibana's "special relationship" with
Elasticsearch. Kibana parses `_reindex`'s `status` field and, because
we're friends with those folks, we should talk to them before we make
backwards breaking changes to it. We *want* to be friends with everyone
but there is only so much time in the day and we don't *want* to make
backwards breaking fields to `status` at all anyway. So we hope that
breaking changes documentation should be enough for other folks.

Relates to #34245.
2018-10-04 14:42:37 -04:00
Benjamin Trent 2dd058d607
HLRC: ML Add preview datafeed api (#34284)
* HLRC: ML Add preview datafeed api

* Changing deprecation handling for parser

* Removing some duplication in docs, will address other APIs in another PR
2018-10-04 11:28:44 -07:00
Nik Everett dc2cf28fde
Docs: Allow skipping response assertions (#34240)
We generate tests from our documentation, including assertions about the
responses returned by a particular API. But sometimes we *can't* assert
that the response is correct because of some defficiency in our tooling.
Previously we marked the response `// NOTCONSOLE` to skip it, but this
is kind of odd because `// NOTCONSOLE` is really to mark snippets that
are json but aren't requests or responses. This introduces a new
construct to skip response assertions:
```
// TESTRESPONSE[skip:reason we skipped this]
```
2018-10-04 08:03:38 -04:00
Ignacio Vera 3ccb7af56a
HLRC: Add activate watch action (#33988)
* HLRC: Add activate watcher action

Adds activate watch action to the high level rest client.

Relates #29827
2018-10-04 12:49:52 +02:00
Ed Savage 577261ee57
[ML] Label anomalies with multi_bucket_impact (#34233)
* [ML] Label anomalies with  multi_bucket_impact

Add the multi_bucket_impact field to record results.
2018-10-04 09:08:21 +01:00
Ioannis Kakavas 74993fe151
Document smtp.ssl.trust configuration option (#34275)
This adds documentation for `smtp.ssl.trust` that was reintroduced
in #31684 in 6.3.2.

Resolves #32936
2018-10-04 10:53:55 +03:00
Ioannis Kakavas 2c82b80b85
Support PKCS#11 tokens as keystores and truststores (#34063)
This enables Elasticsearch to use the JVM-wide configured
PKCS#11 token as a keystore or a truststore for its TLS configuration.
The JVM is assumed to be configured accordingly with the appropriate
Security Provider implementation that supports PKCS#11 tokens.
For the PKCS#11 token to be used as a keystore or a truststore for an
SSLConfiguration, the .keystore.type or .truststore.type must be
explicitly set to pkcs11 in the configuration.
The fact that the PKCS#11 token configuration is JVM wide implies that
there is only one available keystore and truststore that can be used by TLS
configurations in Elasticsearch.
The PIN for the PKCS#11 token can be set as a truststore parameter in
Elasticsearch or as a JVM parameter ( -Djavax.net.ssl.trustStorePassword).

The basic goal of enabling PKCS#11 token support is to allow PKCS#11-NSS in
FIPS mode to be used as a FIPS 140-2 enabled Security Provider.
2018-10-04 10:51:58 +03:00
Kazuhiro Sera d45fe43a68 Fix a variety of typos and misspelled words (#32792) 2018-10-03 18:11:38 +01:00
Mayya Sharipova 8f10c771e6 Add migration info for missing values in script
Relates to #30975
2018-10-03 11:56:18 -04:00
Lee Hinman 90c55f5e36 Merge remote-tracking branch 'origin/master' into index-lifecycle 2018-10-03 09:11:28 -06:00
albendz f09190c14d Require combine and reduce scripts in scripted metrics aggregation (#33452)
* Make text message not required in constructor for slack

* Remove unnecessary comments in test file

* Throw exception when reduce or combine is not provided; update tests

* Update integration tests for scripted metrics to always include reduce and combine

* Remove some old changes from previous branches

* Rearrange script presence checks to be earlier in build

* Change null check order in script builder for aggregated metrics; correct test scripts in IT

* Add breaking change details to PR
2018-10-03 15:22:01 +01:00
Vladimir Dolzhenko a7f62ee902
[GCE Discovery] Automatically set project-id and zone (#33721)
Fetch default values for project-id and zone from metadata server

Closes #13618
2018-10-03 11:37:36 +02:00
Julie Tibshirani c6fcb60071
Add support for 'ack watch' to the HLRC. (#33962) 2018-10-03 02:03:03 -07:00
Christoph Büscher 6fb9c63ed6 [Docs] Fix broken link for HLRC rethrottle 2018-10-02 23:14:37 +02:00
Wilder Pereira 5af6ae564d Change "REST Verb" to "HTTP Verb" (#34195) 2018-10-02 17:09:54 -04:00
Christoph Büscher a1c441f78a
HLRC: Add throttling for update & delete-by-query (#33951)
This change adds throttling to the update-by-query and delete-by-query cases
similar to throttling for reindex. This mostly means additional methods on the
client class itself, since the request hits the same RestHandler, just with
slightly different endpoints, and also the return values are similar.
2018-10-02 21:44:15 +02:00
Gordon Brown fb907706ec Merge branch 'master' into index-lifecycle 2018-10-02 13:43:46 -06:00
Gordon Brown dd3fe92673
[DOCS] Note that User Cluster Metadata is not private (#34156)
As user-defined cluster metadata is accessible to anyone with access to
get the cluster settings, stored in the logs, and likely to be tracked
by monitoring solutions, it is useful to clarify in the documentation
that it should not be used to store secret information.
2018-10-02 13:36:13 -06:00
Andriy 6b714c9e1e [Docs] Updated link to kafka-elasticsearch-consumer project (#34234) 2018-10-02 17:46:38 +02:00
Lisa Cawley a4cf4ca585
[DOCS] Clarifies examples in reindex and task APIs (#33143) 2018-10-02 08:37:45 -07:00
Serge Populov 13af5d5d7f Docs: Fix typo in field name in aggregations (#34223) 2018-10-02 10:54:29 -04:00
Nik Everett f904c41506
HLRC: Add get rollup job (#33921)
Adds support for the get rollup job to the High Level REST Client. I had
to do three interesting and unexpected things:
1. I ported the rollup state wiping code into the high level client
tests. I'll move this into the test framework in a followup and remove
the x-pack version.
2. The `timeout` in the rollup config was serialized using the
`toString` representation of `TimeValue` which produces fractional time
values which are more human readable but aren't supported by parsing. So
I switched it to `getStringRep`.
3. Refactor the xcontent round trip testing utilities so we can test
parsing of classes that don't implements `ToXContent`.
2018-10-02 09:11:29 -04:00