Commit Graph

47725 Commits

Author SHA1 Message Date
James Rodewig 35bf92cdac [DOCS] Reformat enrich stats API (#46600) 2019-09-11 13:52:50 -04:00
Martijn van Groningen 60ad099178
Add HLRC support for enrich get policy API. (#45970)
Changed the signature of AbstractResponseTestCase#createServerTestInstance(...)
to include the randomly selected xcontent type. This is needed for the
creating a server response instance with a query which is represented as BytesReference.
Maybe this should go into a different change?

This PR also includes HLRC docs for the get policy api.

Relates to #32789
2019-09-11 14:42:50 +02:00
Martijn van Groningen a4b0f66919
Add enrich stats api (#46462)
The enrich api returns enrich coordinator stats and
information about currently executing enrich policies.

The coordinator stats include per ingest node:
* The current number of search requests in the queue.
* The total number of outstanding remote requests that
  have been executed since node startup. Each remote
  request is likely to include multiple search requests.
  This depends on how much search requests are in the
  queue at the time when the remote request is performed.
* The number of current outstanding remote requests.
* The total number of search requests that `enrich`
  processors have executed since node startup.

The current execution policies stats include:
* The name of policy that is executing
* A full blow task info object that is executing the policy.

Relates to #32789
2019-09-11 13:40:24 +02:00
Martijn van Groningen b5dbb3f2b3
Merge branch 'enrich_backport_docs_commits' into enrich-7.x 2019-09-11 13:30:52 +02:00
James Rodewig a27d075db4
[DOCS] Update "Enrich your data" tutorials (#46417)
* Move enrich docs to separate file

* Rewrite enrich processor tutorial
2019-09-11 13:08:48 +02:00
James Rodewig d74d995382
[DOCS] Separate Enrich API Docs (#46286)
* Add enrich policy common parameter

* Add enrich APIs to REST APIs index

* Add put enrich policy API docs

* Add get enrich policy API docs

* Add delete enrich policy API docs

* Add execute enrich policy API docs
2019-09-11 13:08:28 +02:00
Martijn van Groningen c79a8e448d
Convert enrich qa modules to use testclusters. 2019-09-11 11:40:18 +02:00
Martijn van Groningen 8a48ef2a06
fixed typo 2019-09-11 09:52:25 +02:00
Martijn van Groningen ef33a99e6e
Disable default features that are not needed for enrich indices. (#46525)
Relates to #32789
2019-09-11 09:20:38 +02:00
Michael Basnight 9304f5c889 Ensure enrich executes on master node only (#46448)
The previous transport action was a read action, which under the right
set of circumstances can execute on a coordinating node. This commit
ensures that cannot happen.
2019-09-10 09:59:36 -05:00
Martijn van Groningen c057fce978
Merge remote-tracking branch 'es/7.x' into enrich-7.x 2019-09-09 08:40:54 +02:00
Nhat Nguyen 24c3a1de3c Ignore replication for noop updates (#46458)
Previously, we ignore replication for noop updates because they do not
have sequence numbers. Since #44603, we started assigning sequence
numbers to noop updates leading them to be replicated to replicas.

This bug occurs only on 8.0 for it requires #41065 and #44603.

Closes #46366
2019-09-07 11:32:01 -04:00
Andrei Stefan 7b26a8c041 Use `null` schema response for `SYS TABLES` command. (#46386)
(cherry picked from commit a6152f42a47a1ccd668e5892778c8bd2d3a78c4c)
2019-09-07 09:24:54 +03:00
Andrei Stefan 7cf100ba07 SQL: fix scripting for grouped by datetime functions (#46421)
* Fix issue with painless scripting not being correctly generated when
datetime functions are used for GROUPing of an INTERVAL operation.

(cherry picked from commit cb92828e8ec9d9d241bd6189e5835fd99f8b9a44)
2019-09-07 09:24:53 +03:00
Armin Braun 1bb1c77885
Increase REST-Test Client Timeout to 60s (#46455) (#46461)
We are seeing requests take more than the default 30s
which leads to requests being retried and returning
unexpected failures like e.g. "index already exists"
because the initial requests that timed out, worked
out functionally anyway.
=> double the timeout to reduce the likelihood of
the failures described in #46091
=> As suggested in the issue, we should in a follow-up
turn off retrying all-together probably
2019-09-07 07:40:16 +02:00
James Rodewig a6cc0deaa0 [DOCS] Remove cat request from Index Segments API requests (#46463) 2019-09-06 16:47:00 -04:00
James Rodewig f04573f8e8
[DOCS] [5 of 5] Change // TESTRESPONSE comments to [source,console-results] (#46449) (#46459) 2019-09-06 16:09:09 -04:00
James Rodewig 61756597ae [DOCS] Correct definition for `allow_no_indices` parameter (#46450) 2019-09-06 14:11:16 -04:00
Anton 6ae1ae9c9a [Docs] Fix typo in field-names-field.asciidoc (#46430) 2019-09-06 18:04:28 +02:00
James Rodewig b05c0f333d [DOCS] Add index alias definition to glossary (#46339) 2019-09-06 11:35:55 -04:00
James Rodewig c46c57d439
[DOCS] Change // CONSOLE comments to [source,console] (#46441) (#46451) 2019-09-06 11:31:13 -04:00
James Rodewig 31b4e2f6df
[DOCS] Resort common-parms (#46419) (#46442) 2019-09-06 10:30:43 -04:00
David Roberts 7c7fb7e32d [ML] Tolerate total_search_time_ms not mapped in get datafeed stats (#46432)
ML users who upgrade from versions prior to 7.4 to 7.4 or later
will have ML results indices that do not have mappings for the
total_search_time_ms field.  Therefore, when searching these
indices we must tolerate this field not having a mapping.

Fixes #46437
2019-09-06 14:31:15 +01:00
Hendrik Muhs c2194aa7e1 [Transform] simplify class structure of indexer (#46306)
simplify transform task and indexer

 - remove redundant transform id
 - moving client data frame indexer (and builder) into a separate file
2019-09-06 15:24:26 +02:00
James Rodewig bb7bff5e30
[DOCS] Replace "// TESTRESPONSE" magic comments with "[source,console-result] (#46295) (#46418) 2019-09-06 09:22:08 -04:00
István Zoltán Szabó 8208ffa666 [DOCS] Adds progress parameter description to the GET stats data frame analytics API doc. (#46434) 2019-09-06 15:18:57 +02:00
markharwood 323ec022be
Deprecate the "index.max_adjacency_matrix_filters" index setting (#46394)
Following performance optimisations to the adjacency_matrix aggregation we no longer require this setting. Marked as deprecated and due for removal in 8.0

Related #46324
2019-09-06 13:59:47 +01:00
Dimitris Athanasiou a6834068e3
[7.x][ML] Extract DataFrameAnalyticsTask into its own class (#46402) (#46426)
This refactors `DataFrameAnalyticsTask` into its own class.
The task has quite a lot of functionality now and I believe it would
make code more readable to have it live as its own class rather than
an inner class of the start action class.

Backport of #46402
2019-09-06 14:13:46 +03:00
lois.左 27889b3d98 Fix typo in update API example (#46397) 2019-09-06 10:31:59 +02:00
Yunfeng,Wu 7582af27b0 Resolve the incorrect scroll_current when delete or close index (#45226)
Resolve the incorrect current scroll for deleted or closed index
2019-09-06 09:45:53 +02:00
Jim Ferenczi f2a6c88f83
Add a system property to ignore awareness attributes (#46375)
This is a follow up of #19191 for 7.x.
This change adds a system property called "es.routing.search_ignore_awareness_attributes" that when set to true will
effectively ignore allocation awareness attributes when routing search and get requests. This is now the default in 8.x so this
commit adds a way to opt-in to this new behavior in a minor version of 7.x.

Relates #45735
2019-09-06 09:29:27 +02:00
Tanguy Leroux 2290865559 Fix usage of randomIntBetween() in testWriteBlobWithRetries (#46380)
This commit fixes the usage of randomIntBetween() in the test 
testWriteBlobWithRetries, when the test generates a random array  
of a single byte.
2019-09-06 09:10:38 +02:00
Hendrik Muhs 42ada88c5d cleanup static member 2019-09-06 08:55:57 +02:00
Hendrik Muhs ab5f09d29b [ML-DataFrame] improve error message for timeout case in stop (#46131)
improve error message if stopping of transform times out.

related #45610
2019-09-06 08:36:01 +02:00
Hendrik Muhs 78824cce81 reuse mock client to avoid probles with thread context closed errors (#46398) 2019-09-06 08:17:25 +02:00
James Rodewig 327da31db4 [DOCS] Reformat index stats API docs (#46322) 2019-09-05 15:45:02 -04:00
Paul Sanwald 758680c549
version bump to 6.8.4 (#46409) 2019-09-05 15:14:36 -04:00
Jason Tedor 92866f977a
Clarify error message on keystore write permissions (#46321)
When the Elasticsearch process does not have write permissions to
upgrade the Elasticsearch keystore, we bail with an error message that
indicates there is a filesystem permissions problem. This commit
clarifies that error message by pointing out the directory where write
permissions are required, or that the user can also run the
elasticsearch-keystore upgrade command manually before starting the
Elasticsearch process. In this case, the upgrade would not be needed at
runtime, so the permissions would not be needed then.
2019-09-05 15:11:54 -04:00
Lisa Cawley 9f1339d0ce [DOCS] Reformats Watcher APIs using template (#46152) 2019-09-05 11:52:23 -07:00
Jason Tedor af1516a377
Add docs on upgrading the keystore (#46331)
This commit adds a note to the docs regarding upgrading the keystore.
2019-09-05 14:41:31 -04:00
Lisa Cawley e078d62247 [DOCS] Identify reloadable Azure repository plugin settings (#46358) 2019-09-05 11:25:26 -07:00
Benjamin Trent 457ff3e2fb
7.x/ml fix instance serialization bwc (#46404)
* [ML] Fixing instance serialization version for bwc

* fixing CppLogMessage
2019-09-05 13:23:26 -05:00
Benjamin Trent d912a49c6f
[7.x] Support geotile_grid aggregation in composite agg sources (#45810) (#46399)
* Support geotile_grid aggregation in composite agg sources (#45810)

Adds support for `geotile_grid` as a source in composite aggs. 

Part of this change includes adding a new docFormat of `GEOTILE` that formats a hashed `long` value into a geotile formatting string `zoom/x/y`.
2019-09-05 13:22:57 -05:00
Lisa Cawley 828ff01515
[DOCS] Update snippets in security APIs (#46191) (#46401) 2019-09-05 11:12:39 -07:00
Benjamin Trent caf3e4d654
[7.x] [ML][Transforms] fixing rolling upgrade continuous transform test (#45823) (#46347) (#46337)
* [ML][Transforms] fixing rolling upgrade continuous transform test (#45823)

* [ML][Transforms] fixing rolling upgrade continuous transform test

* adjusting wait assert logic

* adjusting wait conditions

* [ML][Transforms] allow executor to call start on started task (#46347)

* making sure we only upgrade from 7.4.0 in test
2019-09-05 10:31:11 -05:00
Benjamin Trent bac5cb1c7a
[ML][Transforms] update supported aggs docs (#46388) (#46392) 2019-09-05 10:30:40 -05:00
James Rodewig 96a4dc21e0 [DOCS] Re-add versioning to put template docs (#46384)
Adds documentation for index template versioning
accidentally removed with #46297.
2019-09-05 10:18:36 -04:00
James Rodewig 1f36c4e50c
[DOCS] Replace "// CONSOLE" comments with [source,console] (#46159) (#46332) 2019-09-05 10:11:25 -04:00
James Rodewig d5597c9ba6 [DOCS] Reformat index segments API docs (#46345) 2019-09-05 08:34:59 -04:00
Adrien Grand 3f5f028e2c Fix release version of #38095. 2019-09-05 14:07:00 +02:00