Commit Graph

54907 Commits

Author SHA1 Message Date
James Rodewig 873f4995c5
[DOCS] Fix typo in histogram agg docs (#65822) (#65828) 2020-12-03 10:53:19 -05:00
leonseng 57dca5f44a
Add missing comma in sample payload for the watcher's pagerduty action 2020-12-03 08:29:23 -06:00
Christoph Büscher dc2444c631 Fix lucene_version_path in docs (#65809)
The `lucene_version_path` variable is used in urls pointing to the Lucene docs.
They use underscore separators for the major and minor versions there.
Correcting this since this has been lost in the latest update on the 7.x branch.
2020-12-03 14:51:55 +01:00
Mark Vieira 49d062efb0 Always use "elasticsearch" as Gradle project name (#65709) 2020-12-02 09:29:36 -08:00
Bhavya Gupta dcff07f717
Updated painless-walkthrough documentation (#65530)
Update documentation about regex note.
2020-12-02 09:23:12 -08:00
James Rodewig ff368304a5
[DOCS] Add cluster get settings API example (#65754) (#65758) 2020-12-02 10:56:51 -05:00
James Rodewig d56d2a399b
[DOCS] Remove inert component template file (#65749) (#65752) 2020-12-02 09:59:00 -05:00
James Rodewig 6ae0370dd7
[DOCS] Fix EQL syntax formatting (#65711) (#65744)
Co-authored-by: Howard <danielhuang@tencent.com>
2020-12-02 09:40:52 -05:00
Thiago Souza 5b422dd19d
[DOCS] Remove erroneous `flat_settings` query param (#65670) 2020-12-02 09:15:15 -05:00
István Zoltán Szabó df16045753
[DOCS] Changes wording of pivot parameter in PUT transforms API docs. (#65731) (#65737) 2020-12-02 14:57:25 +01:00
James Rodewig b8291764da
[DOCS] Update reference documentation that mentions CMS (#50542) (#65734)
Relates to https://github.com/elastic/elasticsearch/issues/46973

Co-authored-by: Evgenia Badyanova <evgenia.badiyanova@elastic.co>
2020-12-02 08:11:03 -05:00
Jim Ferenczi 1c34507e66 Create async search index if necessary on updates and deletes (#64606)
This change ensures that we create the async search index with the right mappings and settings when updating or deleting a document. Users can delete the async search index at any time so we have to re-create it internally if necessary before applying any new operation.
2020-12-02 09:04:28 +01:00
James Rodewig bd159d8c17
[DOCS] EQL: Flatten EQL syntax headings (#65693) (#65696) 2020-12-01 13:43:46 -05:00
James Rodewig eca0401d56
[DOCS] EQL: Remove outdated wildcard ref (#65684) (#65691) 2020-12-01 11:56:12 -05:00
Armin Braun 745f527fea
Deduplicate Index Meta Generations when Deserializing (#65619) (#65666)
These strings are quite long individually and will be repeated
potentially up to the number of snapshots in the repository times.
Since these make up more than half of the size of the repository metadata
and are likely the same for all snapshots the savings from deduplicating them
can make up for more than half the size of `RepositoryData` easily in most real-world
cases.
2020-12-01 12:34:35 +01:00
Armin Braun f8f08ba3a7
Fix NPE in ClusterInfoService (#65654) (#65659)
Store stats can be `null` if e.g. the shard was already closed
when the stats where retrieved. Don't record those shards in the
sizes map to fix an NPE in this case.
2020-12-01 10:33:36 +01:00
Armin Braun 16642f1c74
Handle RejectedExecutionException in ShardFollowTasksExecutor (#65648) (#65653)
Follow-up to #65415. We can't have this exception bubble up in an exception
handler any longer due to the new assertion so we must handle it here.
2020-12-01 06:51:05 +01:00
Armin Braun 6bbeedc932
Reset Deflater/Inflater after Use in DeflateCompressor (#65617) (#65646)
We should reset after use, not before reuse. Otherwise we keep the input buffers
on these objects around for a long time and they can grow to O(MB).
2020-12-01 02:44:36 +01:00
Przemko Robakowski bb0fcb150b Fix TranslogTests.testTotalTests when n=0 (#65632)
When n=0 in TranslogTests.testTotalTests we never update earliestLastModifiedAge so it fails comparison with default value of total.getEarliestLastModifiedAge() which is 0.
In this change we always check this special case and then select n>0

Closes #65629
2020-11-30 18:35:55 -05:00
Howard 0137c1679b Fix the earliest last modified age of translog stats (#64753)
Currently translog's `earliest_last_modified_age` field is always 0 in `_nodes/stats` response.
2020-11-30 17:34:55 -05:00
James Rodewig 154c579e9b
[DOCS] Correct restore snapshot API request example (#65525) (#65628)
Co-authored-by: bellengao <gbl_long@163.com>
2020-11-30 14:21:36 -05:00
Dody Suria Wijaya d14dcfc473
[DOCS] Fix type exists API request example (#65574) 2020-11-30 13:45:46 -05:00
Henning Andersen 02ed90b54a Searchable snapshot terminology (#65549)
We chose to use searchable snapshot index over snapshot-backed index, so
changed terminology towards this in a couple places.
2020-11-30 17:17:04 +01:00
Henning Andersen 9564a8b1e0 Cold tier time-range should not be specified (#65546)
Whether the cold tier can handle years depends a lot on the use case and
for instance our BWC guarantees. This would need to be part of a
specific sizing exercise, so in the spirit of not over-promising, the
description of the cold tier has been changed to not mention years.
2020-11-30 17:10:05 +01:00
David Turner aa8ebeb918 Clarify snapshot incrementality (#65587)
Today we describe snapshots as "incremental" but their incrementality is
rather different beast from e.g. incremental filesystem backups. With
traditional backups you take a large and relatively infrequent "full"
backup and then a sequence of smaller "incremental" ones, and this whole
sequence of backups is required for a restore so it must be kept around
until at least the next full backup. In contrast, Elasticsearch
snapshots are logically independent and each can be deleted without
affecting the integrity of the others.

This distinction frequently causes confusion amongst newer users, so
this commit clarifies what we mean by "incremental" in the docs.
2020-11-30 14:58:26 +00:00
James Rodewig a122f10742
[DOCS] Add `require_alias` query param to reindex API (#65608) (#65610) 2020-11-30 09:48:29 -05:00
James Rodewig c13d6082fe
[DOCS] Add missing "with" in remote reindex doc (#65532) (#65603)
Co-authored-by: Dimitris Athanasiou <dimitris@elastic.co>
2020-11-30 09:34:41 -05:00
Alan Woodward fb84b6710d
Restore use of default search and search_quote analyzers (#65491) (#65562)
In the refactoring of TextFieldMapper, we lost the ability to define
a default search or search_quote analyzer in index settings. This
commit restores that ability, and adds some more comprehensive
testing.

Fixes #65434
2020-11-26 18:34:59 +00:00
Ioannis Kakavas f6921af885 Revert "Gracefully handle exceptions from Security Providers (#65464) (#65554)"
This reverts commit 12ba9e3e16. This
commit was mechanically backported to 7.10 while it shouldn't have
been.
2020-11-26 17:11:34 +02:00
Ioannis Kakavas 12ba9e3e16
Gracefully handle exceptions from Security Providers (#65464) (#65554)
In certain situations, such as when configured in FIPS 140 mode,
the Java security provider in use might throw a subclass of
java.lang.Error. We currently do not catch these and as a result
the JVM exits, shutting down elasticsearch.

This commit attempts to address this by catching subclasses of Error
that might be thrown for instance when a PBKDF2 implementation
is used from a Security Provider in FIPS 140 mode, with the password
input being less than 14 bytes (112 bits).

- In our PBKDF2 family of hashers, we catch the Error and
throw an ElasticsearchException while creating or verifying the
hash. We throw on verification instead of simply returning false
on purpose so that the message bubbles up and the cause becomes
obvious (otherwise it would be indistinguishable from a wrong
password).
- In KeyStoreWrapper, we catch the Error in order to wrap and re-throw 
a GeneralSecurityException with a helpful message. This can happen when 
using any of the keystore CLI commands, when the node starts or when we 
attempt to reload secure settings.
- In the `elasticsearch-users` tool, we catch the ElasticsearchException that
the Hasher class re-throws and throw an appropriate UserException.

Tests are missing because it's not trivial to set CI in fips approved mode
right now, and thus any tests would need to be muted. There is a parallel
effort in #64024 to enable that and tests will be added in a followup.
2020-11-26 17:04:34 +02:00
Henning Andersen 9f35b3d402 Clarify searchable snapshot cost trade-offs (#65384)
Clarify that searchable snapshots only result in cost savings for less
frequently accessed data and that the savings do not apply to the entire
cluster.
2020-11-26 13:45:47 +01:00
Ioannis Kakavas b4b4483e24
Do not interpret SecurityException in KeystoreAwareCommand (#65366) (#65486)
KeyStoreAwareCommand attempted to deduce whether an error occurred
because of a wrong password by checking the cause of the
SecurityException that KeyStoreWrapper.decrypt() throws. Checking
for AEADBadTagException was wrong becase that exception could be
(and usually is) wrapped in an IOException. Furthermore, since we
are doing the check already in KeyStoreWrapper, we can just return
the message of the SecurityException to the user directly, as we do
in other places.
2020-11-26 13:12:18 +02:00
Bogdan Pintea 64ddf0834e
Docs: add known issue 65488 - SQL ignoring non-eq (#65492)
* Docs: add known issue 65488 - SQL ignoring non-eq

Add description of the #65488 known issue in release notes.
2020-11-26 11:43:40 +01:00
James Rodewig b3811d2fc2
[DOCS] Add ILM action redirects (#65479) 2020-11-25 11:02:55 -05:00
James Rodewig ff51e8236c
[DOCS] Flatten EQL syntax headings (#65497) (#65504) 2020-11-25 10:50:32 -05:00
Marios Trivyzas 54e7e4c9de
EQL: [Tests] Adjust README for preserving test data (#65460)
Adjusted the README file to mention both the option to preserve the test
data when simple reproducing/executing the tests, but also when starting
the server node manually and issuing the query(ies) against it.

Follows: #65400
(cherry picked from commit e3a1910d28d8b0ed20997754c74fa4d4d52cda15)
2020-11-25 14:30:25 +01:00
richcollier 14cc56de00
Update ml-info-functions.asciidoc
fixed abbreviation for command and control. It is "C2" not "C2C"
2020-11-25 07:44:57 -05:00
Martijn van Groningen 387af748a5
Add support for data stream APIs in transport client. (#65484)
Backporting #65433 to the 7.10 branch.
2020-11-25 10:23:02 +01:00
Martijn van Groningen 4801f7f619
Include the entire response in error message in case of reporting generation error. (#64979)
The toString of HttpResponse includes not just the status, but also all the other details.
2020-11-25 09:24:08 +01:00
James Rodewig f6bcdecbb6 [DOCS] Fix xref 2020-11-24 19:30:56 -05:00
James Rodewig 756c3b8f54 [DOCS] Fix typo 2020-11-24 19:30:26 -05:00
James Rodewig 63d4b1d44b
[DOCS] Fix redirects and anchors (#65472) 2020-11-24 19:23:58 -05:00
Mark Vieira f8f5d27f6b Add option to preserve data in test clusters (#65400)
(cherry picked from commit 1ce323e1368cf5231181f1efaba1c4e425066e37)
2020-11-24 11:56:56 -08:00
Andras Palinkas 7f7e938a25
{S,E}QL: Fix optimization of `NotEquals` in conjunctions (#65331) (#65449)
* Fix the `CombineBinaryComparisons` optimizer rule, so that semantic
equality taken into account during the optimization of `NotEquals`

Examples that previously removed the `NotEquals` expressions (leading
to incorrect results):

```
double >= 10 AND integer != 9
-->  double >= 10

keyword != '2021' AND datetime >= '2020-01-01T00:00:00'
--> datetime >= '2020-01-01T00:00:00'
```

With the fix, expressions like the above will not be touched.
`NotEquals` will only be eliminated from the `AND` expression if the
left side of the `NotEquals` `semanticEquals()` to the left side
of the other expressions within the conjunction (comparisons against
the same field/expression).

* Unit tests and integration tests

Close #65322
(cherry-picked from 8b2b7fa)
2020-11-24 13:20:32 -05:00
James Rodewig be2ed11931
[DOCS] Fix obsolete Slack setup info (#57834) (#65447)
Slack no longer recommends the legacy "integrations" setup (https://api.slack.com/legacy/custom-integrations/incoming-webhooks).  Updated documentation to reference https://api.slack.com/messaging/webhooks instead.  

Removed screenshots from our documentation related to Slack setup.  We should avoid these screenshots (and simply point to Slack documentation) for Slack may change the instructions/their UI in the future.

Also added a short note on the use case of notifying multiple Slack channels.

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-11-24 12:44:46 -05:00
James Rodewig 0c7ee1ff19
[DOCS] Remove 7.10 coming tag (#65440) 2020-11-24 11:40:07 -05:00
Jay Modi 419bda5c15
Fix watcher search template test after #65332 (#65382)
In #65332, the serialization of the WatcherSearchTemplateRequest class
changed to use IndicesOptions built in XContent facilities. This had
the side effect of fixing the handling of `all` for `expand_wildcards`
to include hidden indices. However, the tests in WatcherUtilsTests were
missed. This change updates those tests.

Backport of #65379
2020-11-24 09:04:42 -07:00
James Rodewig 87d74b27af
[DOCS] Correct the url in delete-component-template doc (#65349) (#65427)
Co-authored-by: bellengao <gbl_long@163.com>
2020-11-24 10:07:25 -05:00
Jim Ferenczi 88993e763f Fix handling of null values in geo_point (#65307)
A bug was introduced in 7.10 that causes explicit `null` values to be indexed in the _field_names
field. This change fixes this bug for newly ingested data but `null` values ingested with 7.10 will
continue to match `exists` query so a reindex is required.

Fixes #65306
2020-11-24 11:00:37 +01:00
David Roberts 59ceba8d04 [ML] Adjusting soft_limit description (#65383)
This PR adds detail to the explanation of the soft_limit
memory_status in ML job stats. A consequence that was not
mentioned before is that examples are not added to category
definitions.

Relates elastic/ml-cpp#1590
2020-11-24 09:38:37 +00:00