Commit Graph

49774 Commits

Author SHA1 Message Date
James Rodewig b6bf64b969 [DOCS] Collapse node stats response sections (#51063)
elastic/docs#1687 added support for the `[%collapsible]` Asciidoc
attribute, which creates collapsible sections in the HTML output.

This PR makes two related changes to the nodes stats API documentation:

* Makes the response parameter sections collapsible. This allows users
  to more easily navigate the page without long walls of text.

* Reorders the response parameter sections to match the default order
  returned by the API.

Relates to #47524.
2020-01-16 13:19:29 -05:00
James Rodewig 7ef906fde8 [DOCS] Add tutorials section to analysis topic (#50809)
Adds a 'Configure text analysis' page to house tutorial content for the
analysis topic.

Also relocates the following pages as children as this new page:

* 'Test an analyzer'
* 'Configuring built-in analyzers'
* 'Create a custom analyzer'

I plan to add a tutorial for specifying index-time and search-time
analyzers to this section as part of a future PR.
2020-01-16 13:12:06 -05:00
James Rodewig ef26763ca9 [DOCS] Add concepts section to analysis topic (#50801)
This helps the topic better match the structure of
our machine learning docs, e.g.
https://www.elastic.co/guide/en/machine-learning/7.5/ml-concepts.html

This PR only includes the 'Anatomy of an analyzer' page as a 'Concepts'
child page, but I plan to add other concepts, such as 'Index time vs.
search time', with later PRs.
2020-01-16 13:00:39 -05:00
James Rodewig 1edaf2b101 [DOCS] Retitle analysis reference pages (#51071)
* Changes titles to sentence case.

* Appends pages with 'reference' to differentiate their content from
  conceptual overviews.

* Moves the 'Normalizers' page to end of the Analysis topic pages.
2020-01-16 12:30:51 -05:00
James Rodewig d590150ca2 [DOCS] Fix indent issue in similarity snippet (#51107)
Updates snippet to consistently use 2-space indentation. The snippet
previously used a mix of tab/5-space and 2-space indents.

Co-authored-by: Peter Johnson <wiz@wiz.co.nz>

Co-authored-by: Peter Johnson <peter@geocode.earth>
2020-01-16 11:00:15 -05:00
Zachary Tong e2ca93bad3 Mute GeoGridAggregatorTestCase#testBounds()
Tracking issue: https://github.com/elastic/elasticsearch/issues/51103
2020-01-16 10:28:10 -05:00
Ioannis Kakavas d0554fd317
Fail gracefully on invalid token strings (#51014) (#51096)
When we receive a request with an Authorization header that contains
a Bearer token that is not generated by us or that is malformed in
some way, attempting to decode it as one of our own might cause a
number of exceptions that are not IOExceptions. This commit ensures
that we catch and log these too and call onResponse with `null, so
that we can return 401 instead of 500.

Resolves: #50497
2020-01-16 17:00:17 +02:00
Florian Kelbert 584cb0d926 [DOCS] Correctly read total hits inside watcher config (#50614)
With elastic/elasticsearch#35848, users can now retrieve total hits as an integer when the `rest_total_hits_as_int` query parameter is `true`. This is the default value.

This updates several snippet examples in the Watcher docs that used a workaround to get a total hits integer.
2020-01-16 09:43:25 -05:00
James Rodewig 1211772f6a [DOCS] Use same index in Cluster Allocation Explain docs (#50936)
Updates several example snippets in the Cluster Allocation Explain API
docs to consistently use the `my_index` index.

Previously, the snippets switches from `my_index` to `idx`, which could
confuse users.

Co-authored-by: Emmanuel DEMEY <demey.emmanuel@gmail.com>

Co-authored-by: Emmanuel DEMEY <demey.emmanuel@gmail.com>
2020-01-16 09:15:26 -05:00
Bogdan Pintea fb65ef3f2d
SQL: Extend the optimisations for equalities (#50792) (#51098)
* Extend the optimizations for equalities

This commit supplements the optimisations of equalities in conjunctions
and disjunctions:
* for conjunctions, the existing optimizations with ranges are extended
with not-equalities and inequalities; these lead to a fast resolution,
the conjunction either being evaluate to a FALSE, or the non-equality
conditions being dropped as superfluous;
* optimisations for disjunctions are added to be applied against ranges,
inequalities and not-equalities; these lead to disjunction either
becoming TRUE or the equality being dropped, either as superfluous or
merged into a range/inequality.

* Adress review notes

* Fix the bug around wrongly optimizing 'a=2 OR a!=?', which only yields
TRUE for same values in equality and inequality.
* Var renamings, code style adjustments, comments corrections.

* Address further review comments. Extend optim.

- fix a few code comments;
- extend the Equals OR NotEquals optimitsation (a=2 OR a!=5 -> a!=5);
- extend the Equals OR Range optimisation on limits equality (a=2 OR
  2<=a<5 -> 2<=a<5);
- in case an equality is being removed in a conjunction, the rest of
  possible optimisations to test is now skipped.

* rename one var for better legiblity

- s/rmEqual/removeEquals

(cherry picked from commit 62e7c6a010f10cd7893ee5c99bad8b8d2a693436)
2020-01-16 14:32:34 +01:00
Marios Trivyzas fda25ed04a
Fix caching for PreConfiguredTokenFilter (#50912) (#51091)
The PreConfiguredTokenFilter#singletonWithVersion uses the version
internally for the token filter factories but it registers only one
instance in the cache and not one instance per version. This can lead
to exceptions like the one described in #50734 since the singleton is
created and cached using the version created of the first index
that is processed.

Remove the singletonWithVersion() methods and use the
elasticsearchVersion() methods instead.

Fixes: #50734
(cherry picked from commit 24e1858)
2020-01-16 13:58:02 +01:00
Tom Veasey 32ec934b15
[7.x][ML] Assert top classes are ordered by score (#51028)
Backport #51003.
2020-01-16 12:23:15 +00:00
Ted Timmons b345c7ff31 [Docs] Fix short alias for 'unassigned.for' (#51059)
The short alias for `unassigned.for` is `uf`, not 'ua'.
2020-01-16 12:10:30 +01:00
markharwood ff0a45f882
Fix NPE in PinnedQuery call to DisjunctionMaxScorer. (#51047) (#51064)
Fix NPE in PinnedQuery call to DisjunctionMaxScorer. (#51047)
Added test and fix that tests for score type.
Closes #51034
2020-01-16 10:41:43 +00:00
PND 1d391f7113 [Docs] Fix example output of edge n-gram token filter. (#51085) 2020-01-16 11:34:00 +01:00
Rory Hunter 80d925e225
Auto-format buildSrc (#51043)
Backport / reimplementation of #50786 on 7.x.

Opt-in `buildSrc` for automatic formatting. This required a config tweak
in order to pick up all the Java sources, and as a result more files are
now found in the Enrich plugin, that were previously missed.

I also moved the 2 Java files in `buildSrc/src/main/groovy` into the Java
directory, which required some follow-up changes.
2020-01-16 10:26:27 +00:00
Martijn van Groningen 02dfd71efa
Backport: Add pipeline name to ingest metadata (#51050)
Backport: #50467

This commit adds the name of the current pipeline to ingest metadata.
This pipeline name is accessible under the following key: '_ingest.pipeline'.

Example usage in pipeline:
PUT /_ingest/pipeline/2
{
    "processors": [
        {
            "set": {
                "field": "pipeline_name",
                "value": "{{_ingest.pipeline}}"
            }
        }
    ]
}

Closes #42106
2020-01-16 10:50:47 +01:00
Adrien Grand 45d7bdcfd7
Add analysis components and mapping types to the usage API. (#51062)
Knowing about used analysis components and mapping types would be incredibly
useful in order to know which ones may be deprecated or should get more love.

Some field types also act as a proxy to know about feature usage of some APIs
like the `percolator` or `completion` fields types for percolation and the
completion suggester, respectively.
2020-01-16 09:56:41 +01:00
Tim Vernum ac6602a156
Fix windows newline issue in test (#51082)
Fixes HttpCertificateCommandTests.testTextFileSubstitutions on Windows

Backport of: #51030
2020-01-16 17:01:58 +11:00
Yang Wang c1a6d5d9ff
Encrypt generated key with AES (#51019) (#51076)
Replace DES with AES to align with modern encryption standards
Backport also fixs Files.readString API that is not available in Java 8

Resolves: #50843
2020-01-16 14:47:21 +11:00
Oleksandr Stasyk dd419f0ad0 [DOCS] Add Elixir Bulk Processor to community clients (#50630) 2020-01-15 16:08:56 -05:00
James Rodewig 18d1b7aecd [DOCS] Add note to community clients page (#51065)
Adds a note to the 'Community Contributed Clients' page explicitly
stating that Elastic does not support or endorse the clients.
2020-01-15 15:21:37 -05:00
Zachary Tong a62c9e4e69 Fix 7.6 release notes file name 2020-01-15 15:16:48 -05:00
Zachary Tong 8f48c8d312 Add 7.6.0 release notes 2020-01-15 14:10:37 -05:00
Lee Hinman 2d1c28a45d
[7.x] Fix AllocateRoutedStepTests reusing keys for random valu… (#51058)
In these tests there was a very small chance that keys could collide,
which causes test failures.

Resolves #49307
2020-01-15 11:36:34 -07:00
Lee Hinman e395cf3419
Guard against null settings in CCRIndexLifecycleIT (#51008) (#51054)
It's possible that the index could return no settings and thus throw a
`NullPointerException`.

I wasn't able to reproduce the original issue, but this should guard
against in the future.

Resolves #50646

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-01-15 11:21:18 -07:00
Christoph Büscher d291f189a8
Fix hardcoded version replacement in put-dfanalytics.asciidoc #51053
The version replacement for the code snippet should replace 7.6 with the current version,
but doesn't match because of a missing whitespace.

Closes #51052
2020-01-15 18:09:37 +01:00
taku333 65af0a0f0a [DOCS] Add 7.5.1 link to release notes overview (#51022) 2020-01-15 11:53:26 -05:00
Lee Hinman ad60f0015e
Address failures in SnapshotLifecycleRestIT.testFullPolicySnapshot (#51013)
This test failed a couple of different ways, related to timing, as well
as concurrent snapshots, and also naming.

This commit splits the giant `assertBusy` into separate parts so that we don't
perform ~5 different requests and tests in the same loop. It also gives each
test a unique repository so that no other test can accidentally re-use
snapshots.

Resolves #50358 (hopefully!)
2020-01-15 09:47:41 -07:00
Lee Hinman b14a949fa9 Add blurb about ILM-injected unfollow action (#51009)
These injected actions are harmless and safe to ignore for non-CCR indices.

Resolves #50548
2020-01-15 09:46:57 -07:00
Robin Clarke 5ea18cb4af [Docs] Fix sub-heading in start-stop-ilm.asciidoc (#51045)
Removed superfluous `=`.
2020-01-15 16:16:22 +01:00
Zachary Tong 9eab87062c Bump version to 7.7.0 2020-01-15 10:12:14 -05:00
Yannick Welsch dc47b380c8 Block too many concurrent mapping updates (#51038)
Ensures that there are not too many concurrent dynamic mapping updates going out from the
data nodes to the master.

Closes #50670
2020-01-15 16:02:11 +01:00
Przemysław Witek b4a631277a
Add missing docs for new evaluation metrics (#50967) (#51041) 2020-01-15 15:53:42 +01:00
Armin Braun dd09dc7af6
Fix MultiVersionRepository BwC Tests (#51006)
* Fix MultiVersionRepository BwC Tests

The HLRC doesn't like what its getting back from some older 6.x versions for the restore status
so I moved that request to the low level client.
Closes #50819
2020-01-15 15:19:08 +01:00
Rory Hunter 2f069d8f3f
Tweak formatter config for long generic lines (#51027)
Backport of #50909. The current formatting config allows some long
generic declarations to break the 140 character limit. Tweak the config
to wrap such lines.
2020-01-15 13:17:37 +00:00
Alan Woodward 0257de8c26
Emit warnings when index templates have multiple mappings (#50982)
Index templates created in the 5x line can still be present in the cluster state
through multiple upgrades, and may have more than one mapping defined. 8x
will stop supporting templates with multiple mappings, and we should emit
deprecation warnings in 7x clusters to give users a chance to update their
templates before upgrading.
2020-01-15 11:59:27 +00:00
David Roberts 1536c3e622 [TEST] Increase ML distributed test job open timeout (#50998)
There have been occasional failures, presumably due to
too many tests running in parallel, caused by jobs taking
around 15 seconds to open.  (You can see the job open
successfully during the cleanup phase shortly after the
failure of the test in these cases.)  This change increases
the wait time from 10 seconds to 20 seconds to reduce the
risk of this happening.
2020-01-15 08:58:55 +00:00
Martijn van Groningen e76c3d4d32
Tidy up enrich processors: (#50957)
* Fix generics usages.
* Sealed match processor class.
2020-01-15 08:51:22 +01:00
Tomas Della Vedova 5b6fa79fd8
[ML] Removed key value from the catch regex test (#50977) (#51021) 2020-01-15 08:50:59 +01:00
István Zoltán Szabó b570f417c2 [DOCS] Describes the relationship of the time-related settings in anomaly detection docs (#50959)
Co-Authored-By: David Roberts <dave.roberts@elastic.co>
2020-01-15 08:46:04 +01:00
Tim Vernum e41c0b1224
Deprecating kibana_user and kibana_dashboard_only_user roles (#50963)
This change adds a new `kibana_admin` role, and deprecates
the old `kibana_user` and`kibana_dashboard_only_user`roles.

The deprecation is implemented via a new reserved metadata
attribute, which can be consumed from the API and also triggers
deprecation logging when used (by a user authenticating to
Elasticsearch).

Some docs have been updated to avoid references to these
deprecated roles.

Backport of: #46456

Co-authored-by: Larry Gregory <lgregorydev@gmail.com>
2020-01-15 11:07:19 +11:00
Gordon Brown f207e5bde9
Add generated benchmark files to gitignore (#51000)
When building IntelliJ generates several source files related to the
benchmarks. This commit adds that path to the gitignore so these don't
get accidentally committed.
2020-01-14 16:18:28 -07:00
Nik Everett fc5fde7950
Add "did you mean" to ObjectParser (#50938) (#50985)
Check it out:
```
$ curl -u elastic:password -HContent-Type:application/json -XPOST localhost:9200/test/_update/foo?pretty -d'{
  "dac": {}
}'

{
  "error" : {
    "root_cause" : [
      {
        "type" : "x_content_parse_exception",
        "reason" : "[2:3] [UpdateRequest] unknown field [dac] did you mean [doc]?"
      }
    ],
    "type" : "x_content_parse_exception",
    "reason" : "[2:3] [UpdateRequest] unknown field [dac] did you mean [doc]?"
  },
  "status" : 400
}
```

The tricky thing about implementing this is that x-content doesn't
depend on Lucene. So this works by creating an extension point for the
error message using SPI. Elasticsearch's server module provides the
"spell checking" implementation.
s
2020-01-14 17:53:41 -05:00
Ryan Ernst 4bdab0e985 Fix windows chown to work with single file (#51004)
The chown utility for packaging tests works on windows when the given
path is a directory, but would fail if the path was a single file. This
commit fixes it to handle both cases.

relates #50825
2020-01-14 14:40:04 -08:00
Yannick Welsch 4b0581f182 Remove custom metadata tool (#50813)
Adds a command-line tool to remove broken custom metadata from the cluster state.

Relates to #48701
2020-01-14 23:08:33 +01:00
James Rodewig a290762df1 [DOCS] Document `breakers`, `script`, and `discovery` node stats (#50509)
Documents the `breakers`, `script`, and `discovery` parameters returned
by the `_nodes/stats` API.
2020-01-14 16:51:50 -05:00
Nik Everett a8aca6b2a0
Switch AggregationSpec to ContextParser (#50871) (#50980)
We seem to have settled on the `ContextParser` interface for parsing
stuff, mostly because `ObjectParser` implements it. We don't really
*need* the old `Aggregator.Parser` interface any more because it
duplicates `ContextParser` but with the arguments reversed.

This adds support to `AggregationSpec` to declare aggregation parsers
using `ContextParser`. This should integrate cleanly with
`ObjectParser`. It doesn't drop support for `Aggregator.Parser` or
change the plugin intrface at all so it *should* be safe to backport to
7.x. And we can remove `Aggregator.Parser` in a follow up which is only
targeted to 8.0.
2020-01-14 16:50:52 -05:00
Nik Everett 9a3d4db840
Begin moving date_histogram to offset rounding (backport of #50873) (#50978)
We added a new rounding in #50609 that handles offsets to the start and
end of the rounding so that we could support `offset` in the `composite`
aggregation. This starts moving `date_histogram` to that new offset.
2020-01-14 16:50:27 -05:00
Jason Tedor d5623c8f09
Report progress of multiple plugin installs (#51001)
When installing multiple plugins at once, this commit changes the
behavior to report installed plugins as we go. In the case of failure,
we emit a message that we are rolling back any plugins that were
installed successfully, and also that they were successfully rolled
back. In the case a plugin is not successfully rolled back, we report
this clearly too, alerting the user that there might still be state on
disk they would have to clean up.
2020-01-14 16:24:41 -05:00