Commit Graph

53746 Commits

Author SHA1 Message Date
Julie Tibshirani 9ee5f20ebc Link to the keyword family page from the field types docs. (#61819)
We now link to the top-level keyword type family page instead of its individual
subsections. This better fits the page format, where each type name is a link.
2020-09-01 16:23:49 -07:00
James Rodewig 129b233156
[DOCS] Document dynamic cluster settings (#61760) (#61817)
Co-authored-by: Adam Locke <adam.locke@elastic.co>
2020-09-01 16:04:23 -04:00
Igor Motov 48e53cca94
Fix wrong NaN comparison (#61795) (#61811)
Fixes wrong NaN comparison in error message generator in GeoPolygonDecomposer and PolygonBuilder.

Supersedes #48207

Co-authored-by: Pedro Luiz Cabral Salomon Prado <pedroprado010@users.noreply.github.com>
2020-09-01 15:50:38 -04:00
James Rodewig 8613bde780
[DOCS] Combine keyword family docs (#61662) (#61813) 2020-09-01 15:32:56 -04:00
Rene Groeschke 41613c16e9
Rework output normalization in build-tools func tests (#61706) (#61721) 2020-09-01 21:00:27 +02:00
Tim Brooks e573fa9abc
Add data.path fast path for FilePermission (#61302)
The recursive data.path FilePermission check is an extremely hot
codepath in Elasticsearch. Unfortunately the FilePermission check in
Java is extremely allocation heavy. As it iterates through different
file permissions, it allocates byte arrays for each Path component that
must be compared. This PR improves the situation by adding the recursive
data.path FilePermission it its own PermissionsCollection object which
is checked first.
2020-09-01 12:03:22 -06:00
James Rodewig fd976e668c
[DOCS] EQL: Clarify until keyword docs (#61794) (#61808) 2020-09-01 13:56:51 -04:00
Rene Groeschke 77ecea6e3b
Fix jvm options expansion on windows (#61743) (#61789)
* Fix jvm options expansion on windows
* Avoid long paths on windows
* Relativize paths in jvmoptions against working dir
2020-09-01 19:49:19 +02:00
Armin Braun 28710c985d
Dry up Settings from Map Construction (#61778) (#61803)
We used the same hack all over the place. At least drying it up to a single place.

Co-authored-by: Jay Modi <jaymode@users.noreply.github.com>
2020-09-01 19:46:10 +02:00
Nik Everett f8158bdb2d Skip failing test
Tracked by https://github.com/elastic/elasticsearch/issues/61561
2020-09-01 13:44:31 -04:00
Tanguy Leroux 6e944d9e21
Throws IndexNotFoundException in TransportGetAction for unknown System indices (#61785) (#61791)
The change #57936 introduced a dedicated thread pool for reads in system indices. 
It also introduced a potential NPE in the case the index to read in not yet present in 
the cluster state. This commit fixes that bug by using the getIndexSafe() instead of 
just index() method when retrieving the index's metadata so that an INFE is thrown 
if the index does not exist.
2020-09-01 17:41:57 +02:00
Dimitris Athanasiou 2547cfbe54
[7.x][ML] Persist progress when setting DFA task to failed (#61782) (#61792)
When an error occurs and we set the task to failed via
the `DataFrameAnalyticsTask.setFailed` method we do not
persist progress. If the job is later restarted, this means
we do not correctly restore from where we can but instead
we start the job from scratch and have to redo the reindexing
phase.

This commit solves this bug by persisting the progress before
setting the task to failed.

Backport of #61782
2020-09-01 18:33:07 +03:00
Dan Hermann d52ee17054
Adjust BWC after backport of #60818 2020-09-01 08:30:32 -05:00
Dan Hermann 88a448f1cd
Fix wrong result when executing bulk requests with and without pipeline (#60818) (#61777) 2020-09-01 07:05:25 -05:00
Armin Braun 3fd25bfa87
Fix Concurrent Snapshot Create+Delete + Delete Index (#61770) (#61773)
We had a bug here were we put a `null` value into the shard
assignment mapping when reassigning work after a snapshot delete
had gone through. This only affects partial snaphots but essentially
dead-locks the snapshot process.

Closes #61762
2020-09-01 13:20:25 +02:00
Rory Hunter 50119bb08f Make Docker test robust when removing a temp dir (#61639)
Closes #61614. Closes #61553.

In a particular Docker test, ensure is shut down before removing the
data directory. Also add the force flag to `rm`.
2020-09-01 12:03:41 +01:00
Tanguy Leroux d94d6b5b70 Also account for state not recovered in BlobStoreCacheService
Following #61726 after a test failure
2020-09-01 12:10:04 +02:00
Ioannis Kakavas ced2c140fe
Unmute TokenAuthIntegTests test (#61715)
@ywangd made an awesome analysis on why this test is failing, over
at https://github.com/elastic/elasticsearch/issues/55816#issuecomment-620913282

This change makes it so that we use the same client to perform a
refresh of a token, as we use to subsequently attempt to authenticate
with the refreshed token. This ensures the tests are failing and is
a good approximation of how we expect the same client doing the
refresh, to also perform the subsequent authentication in real life
uses.

The errors we were seeing from users have disappeared after #55114
so we deem our behavior safe.
2020-09-01 13:06:11 +03:00
Tanguy Leroux 787dfda4c1
Prevent snapshots to be mounted as system indices (#61517) (#61727)
System indices can be snapshotted and are therefore potential candidates 
to be mounted as searchable snapshot indices. As of today nothing 
prevents a snapshot to be mounted under an index name starting with . 
and this can lead to conflicting situations because searchable snapshot 
indices are read-only and Elasticsearch expects some system indices 
to be writable; because searchable snapshot indices will soon use an 
internal system index (#60522) to speed up recoveries and we should
prevent the system index to be itself a searchable snapshot index 
(leading to some deadlock situation for recovery).

This commit introduces a changes to prevent snapshots to be mounted 
as a system index.
2020-09-01 11:13:28 +02:00
Tanguy Leroux 92eb6e7844
Remove cluster state listener in BlobStoreCacheService (#61726) (#61769)
BlobStoreCacheService implements ClusterStateListener in order to
 maintain a ready flag that can be used to know when the snapshot 
blob cache should be queries or not.

Now the getAsync() method correctly handles the various exceptions 
that can be thrown when the .snapshot-blob-cache index is not 
available(in isExpectedCacheGetException()) and logs as DEBUG 
we can safely remove the ready flag.
2020-09-01 11:12:52 +02:00
Boice Huang 8fdd3d158b Remove redundant symbol in msearch tests (#61353) 2020-09-01 10:58:22 +02:00
Lisa Cawley d5f1223343 [DOCS] Clarify enabling monitoring features (#61758) 2020-08-31 13:16:23 -07:00
Lisa Cawley 0e4303433b
[DOCS] Document static monitoring settings (#61748) (#61756) 2020-08-31 13:03:17 -07:00
Nik Everett fb84c1f73e
Calculate precise cardinality upper bounds (#61529) (#61754)
This reworks `CardinalityUpperBound` to support precise estimates while
maintaining most of the public API. This will allow us to make more
informed choices about the data structures that we use in aggregations.
None of those interesting choices come as part of this change, but they
are more possible with it.
2020-08-31 15:10:02 -04:00
James Rodewig f39a9bbe19
[DOCS] Document static ILM settings (#61745) (#61749) 2020-08-31 14:02:10 -04:00
Benjamin Trent 7dabaad7d9
[ML] refactor ml job node selection into its own class (#61521) (#61747)
This is a minor refactor where the job node load logic (node availability, etc.) is refactored into its own class.

This will allow future things (i.e. autoscaling decisions) to use the same node load detection class.
backport of #61521
2020-08-31 14:00:23 -04:00
Benjamin Trent 8b33d8813a
[ML] binary classification per-class feature importance for model inference (#61597) (#61746)
This commit addresses two issues:

- per class feature importance is now written out for binary classification (logistic regression)
- The `class_name` in per class feature importance now matches what is written in the `top_classes` array.

backport of https://github.com/elastic/elasticsearch/pull/61597
2020-08-31 13:57:00 -04:00
Dan Hermann 2858e1efc4
Document new stats in _cat/nodes (#60445) (#61742) 2020-08-31 12:40:21 -05:00
James Rodewig 130a7cea78
[DOCS] Add placeholder for 7.9.1 release notes (#61652) 2020-08-31 12:25:59 -04:00
Adam Locke 5723b928d7
Remove Outdated Snapshot Docs (#61684) (#61728)
Removing some now outdated statements that refer to a time
when snapshot operations could not run concurrently.

Closes #61680
2020-08-31 12:04:27 -04:00
Rene Groeschke 4aaeb1e4e3
Address review feedback
- avoid ! when negating boolean expressions
2020-08-31 17:55:16 +02:00
James Rodewig caa1a9024c
[DOCS] Add jump list to breaking changes page (#61598) 2020-08-31 11:25:57 -04:00
James Rodewig f47363074e
[DOCS] Remove placeholder for 7.8.2 release notes (#61653) 2020-08-31 11:22:14 -04:00
James Rodewig 1f24fc03a0
[DOCS] Document dynamic cluster-lvl shard alloc settings (#61338) (#61735) 2020-08-31 11:19:57 -04:00
James Rodewig 8228cdad67
[DOCS] Fix typo in range query docs (#61722) (#61731) 2020-08-31 11:03:11 -04:00
James Rodewig f94999bb2f
[DOCS] Add force merge to hot phase list (#61725) (#61729) 2020-08-31 11:02:41 -04:00
James Rodewig ccbe2938c8
[DOCS] Fix Gsub processor snippet (#61720) (#61723) 2020-08-31 10:43:26 -04:00
Jason Tedor 43cb7c48bd
Adjust Lucene versions for 7.9.1
This commit adjusts the Lucene versions for 7.9.1 after the backporting
of upgrading the 7.9 branch to Lucene 8.6.2.
2020-08-31 10:30:39 -04:00
Mayya Sharipova fe9c66096c Small refactoring of AsyncExecutionId (#61640)
- don't do encoding of asynchExecutionId if it is already provided in
the encoded form
- create a new instance of AsyncExecutionId after checks for
correctness are done
2020-08-31 10:24:36 -04:00
James Rodewig 054a64d66f
[DOCS] Fix old NodeSelector field in Low Level REST Client (#61551) (#61718)
Co-authored-by: Manabu Matsuzaki <matsumana@users.noreply.github.com>
2020-08-31 10:07:58 -04:00
Nhat Nguyen e37ce561c7 Set timeout of auto put-follow request to unbounded (#61679)
If the master node of the follower cluster is busy, then the 
auto-follower will fail to initialize the following process. This also
occurs when an auto-follow pattern matches multiple indices. We should
set the timeout of put-follow requests issued by the auto-follower to
unbounded to avoid this problem.

Closes #56891
2020-08-31 09:58:19 -04:00
Jason Tedor 64cd229b35
Upgrade to Lucene 8.6.2 (#61688)
This commit upgrades the Lucene dependencies to 8.6.2.
2020-08-31 09:54:07 -04:00
James Rodewig e65778c222
[DOCS] Fix typo in nodes stats docs (#61601) (#61717)
Co-authored-by: Henry <henryloh@ucla.edu>
2020-08-31 09:29:50 -04:00
Rene Groeschke f08981c4da
Simplify distribution download configurations (7.x backport) (#61708)
We leverage artifact transforms now when downloading and unpacking elasticsearch distributions.

This has the benefit of

- handcrafted extract tasks on the root project are not required.
- The general tight coupling to the root project has been removed.
- The overall required configurations required to handle a distribution have been reduced
- ElasticsearchDistribution has been simplified by making Extracted an ordinary Configuration
downloaded and unpacked external distributions are reused in later builds by been cached
in the gradle user home.

DistributionDownloadPlugin functional tests have been extended and ported
to DistributionDownloadPluginFuncTest.

* Fix ElasticsearchNode#getDistributionFiles (#61219)

Fixes #61647
2020-08-31 14:36:42 +02:00
Rory Hunter ff6c071275
Implement deprecation logging using log4j (#61629)
Backport of #61474.

Part of #46106. Simplify the implementation of deprecation logging by
relying of log4j more completely, and implementing additional behaviour
through custom appenders and filters.
2020-08-31 12:42:04 +01:00
Henning Andersen 4c9fe31da8 Mute testTooLowConfiguredMemoryStillStarts (#61705)
Related to #61704
2020-08-31 11:19:53 +02:00
Rene Groeschke d1c196a1b2
Minor cleanup 2020-08-31 11:11:32 +02:00
Rene Groeschke 0d488cfbb9
Fix bwc testcluster setup against elastic < 6.3 2020-08-31 11:07:54 +02:00
Armin Braun 5c86b216e8
Fix Race in testGetSnapshotsRequest (#61694) (#61700)
The fact that the data node is already blocked on writing
data files did not guarantee that the cluster state that made
the data node start snapshotting is already applied on master.
This could lead to races where the get snapshots action still
runs based on a state without the snapshot in it, tripping the assertion.
Much safer to handle this by waiting on the non-blocking snapshot create
to return, which guarantees that the CS has been applied on master.

Closes #61541
2020-08-31 11:06:51 +02:00
Armin Braun 0da20579ca
Cleanly Handle S3 SDK Exceptions in Request Counting (#61686) (#61698)
It looks like it is possible for a request to throw an exception early
before any API interaciton has happened. This can lead to the request count
map containing a `null` for the request count key.
The assertion is not correct and we should not NPE here
(as that might also hide the original exception since we are running this code in
a `finally` block from within the S3 SDK).

Closes #61670
2020-08-31 11:05:59 +02:00