Commit Graph

50287 Commits

Author SHA1 Message Date
Nik Everett 5299664ae3
"did you mean" for ObjectParser with top named (#51018) (#51165)
When you declare an ObjectParser with top level named objects like we do
with `significant_terms` we didn't support "did you mean". This fixes
that.

Relates #50938
2020-01-17 12:00:03 -05:00
Ioannis Kakavas 4fc865e579
Don't fallback to anonymous for tokens/apikeys (#51042) (#51159)
This commit changes our behavior so that when we receive a
request with an invalid/expired/wrong access token or API Key
we do not fallback to authenticating as the anonymous user even if
anonymous access is enabled for Elasticsearch.
2020-01-17 18:56:02 +02:00
David Roberts 295665b1ea [ML] Add audit warning for 1000 categories found early in job (#51146)
If 1000 different category definitions are created for a job in
the first 100 buckets it processes then an audit warning will now
be created.  (This will cause a yellow warning triangle in the
ML UI's jobs list.)

Such a large number of categories suggests that the field that
categorization is working on is not well suited to the ML
categorization functionality.
2020-01-17 16:28:45 +00:00
Przemysław Witek da73c9104e
[ML] Fix tests randomly failing on CI (#51142) (#51150) 2020-01-17 14:58:58 +01:00
Dimitris Athanasiou b70ebdeb96
[7.x][ML] DF Analytics _explain API should skip object fields (#51115) (#51147)
Object fields cannot be used as features. At the moment _explain
API includes them and even worse it allows it does not error when
an object field is excluded. This creates the expectation to the
user that all children fields will also be excluded while it's not
the case.

This commit omits object fields from the _explain API and also
adds an error if an object field is included or excluded.

Backport of #51115
2020-01-17 14:02:59 +02:00
Hendrik Muhs 8282744207 increase timeout of DocsClientYamlTestSuiteIT
increase timeout of DocsClientYamlTestSuiteIT to 35 minutes, temporary solution for issue #49753
2020-01-17 12:58:30 +01:00
Przemysław Witek b1a526d5e9
[7.x] [ML] Update DFA progress document in the index the document belongs to (#51111) (#51117) 2020-01-17 08:12:54 +01:00
Hendrik Muhs 13343b15c9 [Transform] Improve force stop robustness in case of an error (#51072)
If a transform config got lost (e.g. because the internal index disappeared) tasks could not be
stopped using transform API. This change makes it possible to stop transforms without a config,
meaning to remove the background task. In order to do so force must be set to true.
2020-01-17 07:42:21 +01:00
Mark Vieira fac509836a
Upgrade to Gradle 6.1 (#50453)
(cherry picked from commit bdd7bda47b4aa14a14273f7301d3615f1cc4bba0)
2020-01-16 13:58:29 -08:00
Armin Braun c794803e3d
Fix X-Pack Indices Breaking Repository BwC Tests (#51120) (#51125)
If some internal `.watcher` or so index gets created during these tests
then the shard counts on snapshot restores and creates won't match up with expectations.
Fixed by only creating the snapshot for the test index

Closes #50819
2020-01-16 21:52:57 +01:00
Armin Braun e51b209dd3
Fix Infinite Retry Loop in loading RepositoryData (#50987) (#51093)
* Fix Infinite Retry Loop in loading RepositoryData

We were running into an infinite loop when trying to load corrupted (or otherwise un-loadable)
repository data for a repo that uses best effort consistency (e.g. that was just freshly mounted
as done in the test) because we kepy resetting to `-1` on `IOException`, listing and finding the broken
generation `N` and then interpreted the subsequent reset to `-1` as a concurrent change to the repository.
2020-01-16 21:08:35 +01:00
Nik Everett f6c89b4599
Move test of custom sig heuristic to plugin (#50891) (#51067)
This moves the testing of custom significance heuristic plugins from an
`ESIntegTestCase` to an example plugin. This is *much* more "real" and
can be used as an example for anyone that needs to actually build such a
plugin. The old test had testing concerns and the example all jumbled
together.
2020-01-16 14:49:12 -05:00
James Rodewig 2353fe47fc [DOCS] Adds placeholder for 7.5.2 release notes (#51124)
Co-Authored-By: Lisa Cawley <lcawley@elastic.co>
2020-01-16 14:42:24 -05:00
James Rodewig b6bf64b969 [DOCS] Collapse node stats response sections (#51063)
elastic/docs#1687 added support for the `[%collapsible]` Asciidoc
attribute, which creates collapsible sections in the HTML output.

This PR makes two related changes to the nodes stats API documentation:

* Makes the response parameter sections collapsible. This allows users
  to more easily navigate the page without long walls of text.

* Reorders the response parameter sections to match the default order
  returned by the API.

Relates to #47524.
2020-01-16 13:19:29 -05:00
James Rodewig 7ef906fde8 [DOCS] Add tutorials section to analysis topic (#50809)
Adds a 'Configure text analysis' page to house tutorial content for the
analysis topic.

Also relocates the following pages as children as this new page:

* 'Test an analyzer'
* 'Configuring built-in analyzers'
* 'Create a custom analyzer'

I plan to add a tutorial for specifying index-time and search-time
analyzers to this section as part of a future PR.
2020-01-16 13:12:06 -05:00
James Rodewig ef26763ca9 [DOCS] Add concepts section to analysis topic (#50801)
This helps the topic better match the structure of
our machine learning docs, e.g.
https://www.elastic.co/guide/en/machine-learning/7.5/ml-concepts.html

This PR only includes the 'Anatomy of an analyzer' page as a 'Concepts'
child page, but I plan to add other concepts, such as 'Index time vs.
search time', with later PRs.
2020-01-16 13:00:39 -05:00
James Rodewig 1edaf2b101 [DOCS] Retitle analysis reference pages (#51071)
* Changes titles to sentence case.

* Appends pages with 'reference' to differentiate their content from
  conceptual overviews.

* Moves the 'Normalizers' page to end of the Analysis topic pages.
2020-01-16 12:30:51 -05:00
James Rodewig d590150ca2 [DOCS] Fix indent issue in similarity snippet (#51107)
Updates snippet to consistently use 2-space indentation. The snippet
previously used a mix of tab/5-space and 2-space indents.

Co-authored-by: Peter Johnson <wiz@wiz.co.nz>

Co-authored-by: Peter Johnson <peter@geocode.earth>
2020-01-16 11:00:15 -05:00
Zachary Tong e2ca93bad3 Mute GeoGridAggregatorTestCase#testBounds()
Tracking issue: https://github.com/elastic/elasticsearch/issues/51103
2020-01-16 10:28:10 -05:00
Ioannis Kakavas d0554fd317
Fail gracefully on invalid token strings (#51014) (#51096)
When we receive a request with an Authorization header that contains
a Bearer token that is not generated by us or that is malformed in
some way, attempting to decode it as one of our own might cause a
number of exceptions that are not IOExceptions. This commit ensures
that we catch and log these too and call onResponse with `null, so
that we can return 401 instead of 500.

Resolves: #50497
2020-01-16 17:00:17 +02:00
Florian Kelbert 584cb0d926 [DOCS] Correctly read total hits inside watcher config (#50614)
With elastic/elasticsearch#35848, users can now retrieve total hits as an integer when the `rest_total_hits_as_int` query parameter is `true`. This is the default value.

This updates several snippet examples in the Watcher docs that used a workaround to get a total hits integer.
2020-01-16 09:43:25 -05:00
James Rodewig 1211772f6a [DOCS] Use same index in Cluster Allocation Explain docs (#50936)
Updates several example snippets in the Cluster Allocation Explain API
docs to consistently use the `my_index` index.

Previously, the snippets switches from `my_index` to `idx`, which could
confuse users.

Co-authored-by: Emmanuel DEMEY <demey.emmanuel@gmail.com>

Co-authored-by: Emmanuel DEMEY <demey.emmanuel@gmail.com>
2020-01-16 09:15:26 -05:00
Bogdan Pintea fb65ef3f2d
SQL: Extend the optimisations for equalities (#50792) (#51098)
* Extend the optimizations for equalities

This commit supplements the optimisations of equalities in conjunctions
and disjunctions:
* for conjunctions, the existing optimizations with ranges are extended
with not-equalities and inequalities; these lead to a fast resolution,
the conjunction either being evaluate to a FALSE, or the non-equality
conditions being dropped as superfluous;
* optimisations for disjunctions are added to be applied against ranges,
inequalities and not-equalities; these lead to disjunction either
becoming TRUE or the equality being dropped, either as superfluous or
merged into a range/inequality.

* Adress review notes

* Fix the bug around wrongly optimizing 'a=2 OR a!=?', which only yields
TRUE for same values in equality and inequality.
* Var renamings, code style adjustments, comments corrections.

* Address further review comments. Extend optim.

- fix a few code comments;
- extend the Equals OR NotEquals optimitsation (a=2 OR a!=5 -> a!=5);
- extend the Equals OR Range optimisation on limits equality (a=2 OR
  2<=a<5 -> 2<=a<5);
- in case an equality is being removed in a conjunction, the rest of
  possible optimisations to test is now skipped.

* rename one var for better legiblity

- s/rmEqual/removeEquals

(cherry picked from commit 62e7c6a010f10cd7893ee5c99bad8b8d2a693436)
2020-01-16 14:32:34 +01:00
Marios Trivyzas fda25ed04a
Fix caching for PreConfiguredTokenFilter (#50912) (#51091)
The PreConfiguredTokenFilter#singletonWithVersion uses the version
internally for the token filter factories but it registers only one
instance in the cache and not one instance per version. This can lead
to exceptions like the one described in #50734 since the singleton is
created and cached using the version created of the first index
that is processed.

Remove the singletonWithVersion() methods and use the
elasticsearchVersion() methods instead.

Fixes: #50734
(cherry picked from commit 24e1858)
2020-01-16 13:58:02 +01:00
Tom Veasey 32ec934b15
[7.x][ML] Assert top classes are ordered by score (#51028)
Backport #51003.
2020-01-16 12:23:15 +00:00
Ted Timmons b345c7ff31 [Docs] Fix short alias for 'unassigned.for' (#51059)
The short alias for `unassigned.for` is `uf`, not 'ua'.
2020-01-16 12:10:30 +01:00
markharwood ff0a45f882
Fix NPE in PinnedQuery call to DisjunctionMaxScorer. (#51047) (#51064)
Fix NPE in PinnedQuery call to DisjunctionMaxScorer. (#51047)
Added test and fix that tests for score type.
Closes #51034
2020-01-16 10:41:43 +00:00
PND 1d391f7113 [Docs] Fix example output of edge n-gram token filter. (#51085) 2020-01-16 11:34:00 +01:00
Rory Hunter 80d925e225
Auto-format buildSrc (#51043)
Backport / reimplementation of #50786 on 7.x.

Opt-in `buildSrc` for automatic formatting. This required a config tweak
in order to pick up all the Java sources, and as a result more files are
now found in the Enrich plugin, that were previously missed.

I also moved the 2 Java files in `buildSrc/src/main/groovy` into the Java
directory, which required some follow-up changes.
2020-01-16 10:26:27 +00:00
Martijn van Groningen 02dfd71efa
Backport: Add pipeline name to ingest metadata (#51050)
Backport: #50467

This commit adds the name of the current pipeline to ingest metadata.
This pipeline name is accessible under the following key: '_ingest.pipeline'.

Example usage in pipeline:
PUT /_ingest/pipeline/2
{
    "processors": [
        {
            "set": {
                "field": "pipeline_name",
                "value": "{{_ingest.pipeline}}"
            }
        }
    ]
}

Closes #42106
2020-01-16 10:50:47 +01:00
Adrien Grand 45d7bdcfd7
Add analysis components and mapping types to the usage API. (#51062)
Knowing about used analysis components and mapping types would be incredibly
useful in order to know which ones may be deprecated or should get more love.

Some field types also act as a proxy to know about feature usage of some APIs
like the `percolator` or `completion` fields types for percolation and the
completion suggester, respectively.
2020-01-16 09:56:41 +01:00
Tim Vernum ac6602a156
Fix windows newline issue in test (#51082)
Fixes HttpCertificateCommandTests.testTextFileSubstitutions on Windows

Backport of: #51030
2020-01-16 17:01:58 +11:00
Yang Wang c1a6d5d9ff
Encrypt generated key with AES (#51019) (#51076)
Replace DES with AES to align with modern encryption standards
Backport also fixs Files.readString API that is not available in Java 8

Resolves: #50843
2020-01-16 14:47:21 +11:00
Oleksandr Stasyk dd419f0ad0 [DOCS] Add Elixir Bulk Processor to community clients (#50630) 2020-01-15 16:08:56 -05:00
James Rodewig 18d1b7aecd [DOCS] Add note to community clients page (#51065)
Adds a note to the 'Community Contributed Clients' page explicitly
stating that Elastic does not support or endorse the clients.
2020-01-15 15:21:37 -05:00
Zachary Tong a62c9e4e69 Fix 7.6 release notes file name 2020-01-15 15:16:48 -05:00
Zachary Tong 8f48c8d312 Add 7.6.0 release notes 2020-01-15 14:10:37 -05:00
Lee Hinman 2d1c28a45d
[7.x] Fix AllocateRoutedStepTests reusing keys for random valu… (#51058)
In these tests there was a very small chance that keys could collide,
which causes test failures.

Resolves #49307
2020-01-15 11:36:34 -07:00
Lee Hinman e395cf3419
Guard against null settings in CCRIndexLifecycleIT (#51008) (#51054)
It's possible that the index could return no settings and thus throw a
`NullPointerException`.

I wasn't able to reproduce the original issue, but this should guard
against in the future.

Resolves #50646

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-01-15 11:21:18 -07:00
Christoph Büscher d291f189a8
Fix hardcoded version replacement in put-dfanalytics.asciidoc #51053
The version replacement for the code snippet should replace 7.6 with the current version,
but doesn't match because of a missing whitespace.

Closes #51052
2020-01-15 18:09:37 +01:00
taku333 65af0a0f0a [DOCS] Add 7.5.1 link to release notes overview (#51022) 2020-01-15 11:53:26 -05:00
Lee Hinman ad60f0015e
Address failures in SnapshotLifecycleRestIT.testFullPolicySnapshot (#51013)
This test failed a couple of different ways, related to timing, as well
as concurrent snapshots, and also naming.

This commit splits the giant `assertBusy` into separate parts so that we don't
perform ~5 different requests and tests in the same loop. It also gives each
test a unique repository so that no other test can accidentally re-use
snapshots.

Resolves #50358 (hopefully!)
2020-01-15 09:47:41 -07:00
Lee Hinman b14a949fa9 Add blurb about ILM-injected unfollow action (#51009)
These injected actions are harmless and safe to ignore for non-CCR indices.

Resolves #50548
2020-01-15 09:46:57 -07:00
Robin Clarke 5ea18cb4af [Docs] Fix sub-heading in start-stop-ilm.asciidoc (#51045)
Removed superfluous `=`.
2020-01-15 16:16:22 +01:00
Zachary Tong 9eab87062c Bump version to 7.7.0 2020-01-15 10:12:14 -05:00
Yannick Welsch dc47b380c8 Block too many concurrent mapping updates (#51038)
Ensures that there are not too many concurrent dynamic mapping updates going out from the
data nodes to the master.

Closes #50670
2020-01-15 16:02:11 +01:00
Przemysław Witek b4a631277a
Add missing docs for new evaluation metrics (#50967) (#51041) 2020-01-15 15:53:42 +01:00
Armin Braun dd09dc7af6
Fix MultiVersionRepository BwC Tests (#51006)
* Fix MultiVersionRepository BwC Tests

The HLRC doesn't like what its getting back from some older 6.x versions for the restore status
so I moved that request to the low level client.
Closes #50819
2020-01-15 15:19:08 +01:00
Rory Hunter 2f069d8f3f
Tweak formatter config for long generic lines (#51027)
Backport of #50909. The current formatting config allows some long
generic declarations to break the 140 character limit. Tweak the config
to wrap such lines.
2020-01-15 13:17:37 +00:00
Alan Woodward 0257de8c26
Emit warnings when index templates have multiple mappings (#50982)
Index templates created in the 5x line can still be present in the cluster state
through multiple upgrades, and may have more than one mapping defined. 8x
will stop supporting templates with multiple mappings, and we should emit
deprecation warnings in 7x clusters to give users a chance to update their
templates before upgrading.
2020-01-15 11:59:27 +00:00