Commit Graph

48554 Commits

Author SHA1 Message Date
Ioannis Kakavas 24e43dfa34
[7.x] Refactor FIPS BootstrapChecks to simple checks (#47499) (#48333)
FIPS 140 bootstrap checks should not be bootstrap checks as they
are always enforced. This commit moves the validation logic within
the security plugin.
The FIPS140SecureSettingsBootstrapCheck was not applicable as the
keystore was being loaded on init, before the Bootstrap checks
were checked, so an elasticsearch keystore of version < 3 would
cause the node to fail in a FIPS 140 JVM before the bootstrap check
kicked in, and as such hasn't been migrated.

Resolves: #34772
2019-10-22 12:49:01 +03:00
Przemysław Witek aa29567e11
[7.x] Fix assignment
Backport of https://github.com/elastic/elasticsearch/pull/48216
2019-10-22 11:34:09 +02:00
Alpar Torok 144e8a0f1a Use an env var for the classpath of jar hell task (#48240)
The classpath for some project could outgrow the max allowed command
line on Windows. Using an env var is not fault proof, but give more
breathing room
2019-10-22 11:17:19 +03:00
Andrei Stefan 3233b59b68 Add "format" to "range" queries resulted from optimizing a logical AND (#48073)
(cherry picked from commit 020939a9bd5b34c6d540faa8b3a67b740d661be3)
2019-10-22 10:17:37 +03:00
Hendrik Muhs 0656207e9a [DOCS][Transform] document limitation regarding rolling upgrade with 7.2, 7.3 (#48118)
adds a limitation about rolling upgrade from 7.2 or 7.3. and fixes a problem with renamed preferences
2019-10-22 09:01:45 +02:00
Hendrik Muhs 1cb3b0cc0d
[7.6][Transform] separate old and mixed rolling upgrade tests (#48302)
separates rolling upgrade tests for transforms created on old and mixed clusters and disable
testing transforms on mixed clusters for <7.4.
2019-10-22 08:58:02 +02:00
Martijn van Groningen bbe50eca72
Fail with a better error when if there are no ingest nodes (#48272)
when executing enrich execute policy api.
2019-10-22 07:42:04 +02:00
Martijn van Groningen 0ec0ab64c9
Fix executing enrich policies stats (#48132)
The enrich stats api picked the wrong task to be displayed
in the executing stats section.

In case `wait_for_completion` was set to `false` then no task
was being displayed and if that param was set to `true` then
the wrong task was being displayed (transport action task instead
of enrich policy executor task).

Testing executing policies in enrich stats api is tricky.
I have verified locally that this commit fixes the bug.
2019-10-22 07:41:56 +02:00
Martijn van Groningen c09b62d5bf
Backport: also validate source index at put enrich policy time (#48311)
Backport of: #48254

This changes tests to create a valid
source index prior to creating the enrich policy.
2019-10-22 07:38:16 +02:00
Nhat Nguyen d0a4bad95b Use MultiFileTransfer in CCR remote recovery (#44514)
Relates #44468
2019-10-21 23:30:52 -04:00
Tim Brooks 547e399dbf
Remove option to enable direct buffer pooling (#48310)
This commit removes the option to change the netty system properties to
reenable the direct buffer pooling. It also removes the need for us to
disable the buffer pooling in the system properties file. Instead, we
programmatically craete an allocator that is used by our networking
layer.

This commit does introduce an Elasticsearch property which allows the
user to fallback on the netty default allocator. If they choose this
option, they can configure the default allocator how they wish using the
standard netty properties.
2019-10-21 19:15:50 -06:00
James Baiera 0d12ef8958
Add Enrich Origin (#48098) (#48312)
This PR adds an origin for the Enrich feature, and modifies the background 
maintenance task to use the origin when executing client operations. 
Without this fix, the maintenance task fails to execute when security is 
enabled.
2019-10-21 16:40:49 -04:00
Przemysław Witek 2db2b945ec
[7.x] Change format of MulticlassConfusionMatrix result to be more self-explanatory (#48174) (#48294) 2019-10-21 22:07:19 +02:00
Mark Vieira 178204703a
Add 'javadoc' task to lifecycle check tasks (#48214) 2019-10-21 09:45:01 -07:00
James Rodewig f4fa61b2f2 [DOCS] Add 'Selecting gateway and seed nodes' section to CCS docs (#48297) 2019-10-21 11:14:23 -05:00
Armin Braun e65c60915a
Cleanup FileRestoreContext Abstractions (#48173) (#48300)
This class is only used by the blob store repository
and CCR and the abstractions didn't really make sense
with CCR ignoring the concrete `restoreFiles` method
completely and having a method used only by the blobstore
overriden as unsupported.
=> Moved to a more fitting set of abstractions
=> Dried up the stream wrapping in `BlobStoreRepository` a little
now that the `restoreFile` method could be simplified

Relates #48110 as it makes changing the API of `FileRestoreContext`
to what is needed for async restores simpler
2019-10-21 17:30:35 +02:00
Lee Hinman cc0c876a8d
fix incorrect comparison (#48208) (#48303)
* remove comparison of identical values

the comparison `tookInMillis == tookInMillis` is always true.

* add comparison between tookInMillis
2019-10-21 09:14:44 -06:00
Armin Braun dc08feadc6
Remove Redundant Version Param from Repository APIs (#48231) (#48298)
This parameter isn't used by any implementation
2019-10-21 16:20:45 +02:00
Alpar Torok cdfac9dfb0 Don't build packages on non Linux (#48246)
* Don't build packages on non Linux

Closes #47007

* Explicitly exclude windows only
2019-10-21 17:09:27 +03:00
Alpar Torok 5729c04df2 Fix link to GCP upload in build scans (#48248)
* Fix link to GCP upload in build scans

Also changed the name of the link to make it less confusing

* Change the name
2019-10-21 17:04:24 +03:00
James Rodewig a66bb2c7ed [DOCS] Reformat CJK bigram and CJK width token filter docs (#48210) 2019-10-21 08:44:49 -05:00
Marios Trivyzas 2751a4ff1b [Docs] Fix testing docs regarding --debug-jvm (#48293)
Remove extra whitespace and highlight the starting port number.

Follows: #48188
(cherry picked from commit b84b7e13c888c9a4b110c436b4eec1ffd0aebd52)
2019-10-21 15:21:18 +02:00
Benjamin Trent abd1b5118f
[ML] fixing tests (#48084) (#48253)
* [ML] fixing tests

* unmuting tests

* reverting outlier detection job changes
2019-10-21 09:21:06 -04:00
Tanguy Leroux 0094bd5939
Fix AutoFollowIT.testPauseAndResumeWithMultipleAutoFollowPatterns (#48289)
The test testPauseAndResumeWithMultipleAutoFollowPatterns
failed multiple times, mostly because it creates too many leader
indices and the following cluster cannot cope with cluster state
updates generated by following indices creation and pause/
resume auto-followers changes.

This commit simplifies the test by creating at most 20 leader
indices and by waiting for any new leader index to be picked
up by the auto-follower before created another leader index.
It also pause and resume less auto-followers as previously.

closes #47917
2019-10-21 14:31:58 +02:00
Alpar Torok 7d085ffbd9 Switch to debug with server=n (#48188)
Before this change one needed to re-start debugging several times, as we
launched multiple JVMs in debug mode.
With this option the IDE has the option to re-launch and listen for
connections again leading for to a more pleasant experience.
2019-10-21 15:07:04 +03:00
Przemysław Witek 1a42e37070
[7.x] Default "prediction_field_name" to (dependent_variable + "_prediction") (#48232) (#48279) 2019-10-21 13:18:08 +02:00
Albert Zaharovits 69fc715bc3
Fix security origin for TokenService#findActiveTokensFor... (#47418) (#48280)
All internal searches (triggered by APIs) across the .security index
must be performed while "under the security origin". Otherwise,
the search is performed in the context of the caller which most
likely does not have privileges to search .security (hopefully).
This commit fixes this in the case of two methods in the
TokenService and corrects an overly done such context switch
in the ApiKeyService.

In addition, this makes all tests from the client/rest-high-level
module execute as an all mighty administrator,
but not a literal superuser.

Closes #47151
2019-10-21 13:15:05 +03:00
Alpar Torok fe265f0308 Mute CliSecurityIT
tracking in #48117
2019-10-21 11:25:56 +03:00
David Turner 672b2a92ca Fix compile error from previous commit (#48230)
The previous commit, 3a6fa0bbdb introduces a
compile error that was fixed locally but not committed. This commit adds the
missing change.
2019-10-21 08:54:04 +01:00
David Turner 3a6fa0bbdb Close query cache on index service creation failure (#48230)
Today it is possible that we create the `QueryCache` and then fail to create
the owning `IndexService` and this means we do not close the `QueryCache`
again. This commit addresses that leak.

Fixes #48186
2019-10-21 08:46:53 +01:00
Alpar Torok c903ac2376
Fix download of 6.x rpm and deb packages (#48228) 2019-10-21 10:03:44 +03:00
Ignacio Vera b1224fca8c
upgrade to Lucene-8.3.0-snapshot-25968e3b75e (#48227) 2019-10-21 08:21:09 +02:00
Hendrik Muhs 4b3a672a0b
blacklist transform tests in mixed cluster prior 7.4 (#48247)
blacklist transform tests in mixed cluster prior 7.4, see #48019

fixes #47943
2019-10-21 08:19:58 +02:00
Gordon Brown a2217f4a91
Fix testRetentionWhileSnapshotInProgress (#48219)
This test could fail for two reasons, both should be fixed by this PR:

1) It hit a timeout for an `assertBusy`. This commit increases the
timeout for that `assertBusy`.

2) The snapshot that was supposed to be blocked could, in fact, be
successful. This is because a previous snapshot had been successfully
been taken, and no new data had been added between the two snapshots.
This means that no new segment files needed to be written for the new
snapshot, so the block on data files was never triggered. This commit
changes two things: First, it indexes some new data before taking the
second snapshot (the one that needs to be blocked), and second,
checks to ensure that the block is actually hit before continuing
with the test.
2019-10-18 14:25:18 -06:00
Christoph Büscher 3ea666d694 Clarify mapping types that support ignore_malformed (#48206)
The `ignore_malformed` setting only works on selected mapping types, otherwise
we throw an mapper_parsing_exception. We should add a list of all the mapping
types that support it, since the number of types not supporting it seems larger.

Closes #47166
2019-10-18 20:39:38 +02:00
Alpar Torok cc26e30281
Increase timeout for yml tests (#48237)
Some of these are larger than what can complete in the regular timeout.

Closes #48212
2019-10-18 11:14:15 -07:00
Mark Vieira 0e24e6fdd1
Upgrade to build scan plugin 2.4.2 (#48220) 2019-10-18 11:04:52 -07:00
Martijn van Groningen 403728dc3d
Use correct policy type in test 2019-10-18 16:58:03 +02:00
Takuya Kajiwara a56daeae2d [DOCS] Fix typos in InternalEngine.java comments (#46861) 2019-10-18 10:36:58 -04:00
Guido Lena Cota 1beeb3a83e [DOCS] Fix typos in Painless execute API docs(#47518) 2019-10-18 10:31:04 -04:00
Martijn van Groningen 5cb44a414c
Fixed links in java docs for EnrichClient (#48233) 2019-10-18 16:26:49 +02:00
Alexander Reelsen 66581d8158
update ingest-user-agent regexes.yml (#47807)
This new regexes are from:
154eba17f5/regexes.yaml
2019-10-18 16:26:48 +02:00
Martijn van Groningen 844825a13f
Validate policy type when storing an enrich policy (#48126) 2019-10-18 16:26:48 +02:00
James Rodewig 1d06be2f05 [DOCS] Note clause limit in `index.mapping.total_fields.limit` docs (#48153)
Notes that you should increase the `indices.query.bool.max_clause_count`
setting if you increase the `index.mapping.total_fields.limit`
setting.

Closes #46928.
2019-10-18 10:20:49 -04:00
markharwood fbd43196a8 [DOCS] Add TermVectors API reference to MLT query docs (#37228)
Adds a reference the use of the TermVectors API following an issue raised about lack of flexibility in the MLT query: https://github.com/elastic/elasticsearch/issues/35509
2019-10-18 10:08:57 -04:00
Benjamin Trent 876f4aafac
[ML] Add logistic_regression output aggregator (#48238) (#48244) 2019-10-18 10:08:17 -04:00
David Turner a8bcbbc38a Quieter logging from the DiskThresholdMonitor (#48115)
Today if an Elasticsearch node reaches a disk watermark then it will repeatedly
emit logging about it, which implies that some action needs to be taken by the
administrator. This is misleading. Elasticsearch strives to keep nodes under
the high watermark, but it is normal to have a few nodes occasionally exceed
this level. Nodes may be over the low watermark for an extended period without
any ill effects.

This commit enhances the logging emitted by the `DiskThresholdMonitor` to be
less misleading. The expected case of hitting the high watermark and
immediately relocating one or more shards that to bring the node back under the
watermark again is reduced in severity to `INFO`. Additionally, `INFO` messages
are not emitted repeatedly.

Fixes #48038
2019-10-18 15:00:14 +01:00
rsarawgi 5e4dd0fd2e [ML] Removing usages of ToXContentParams.INCLUDE_TYPE (#48165)
Removing the option of ToXContentParams.INCLUDE_TYPE and replacing them with ToXContentParams.FOR_INTERNAL_STORAGE
Closes #48057
2019-10-18 14:49:26 +01:00
David Turner 6a97aef8ac Clarify contribution instructions re. license hdrs (#48067)
Today our instructions in `CONTRIBUTING.md` regarding license headers are a
little unclear. This commit clarifies the exact headers that are required for
contributions.
2019-10-18 14:45:03 +01:00
Alpar Torok a675b35680 Mute org.elasticsearch.xpack.sql.jdbc.JdbcConfigurationTests.testDriverConfigurationWithSSLInURL
tracked in #41557
2019-10-18 11:05:56 +01:00