Commit Graph

1082 Commits

Author SHA1 Message Date
Nik Everett a9b72019ad Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@b45f682d72
2017-08-01 16:17:17 -04:00
Jay Modi a35234de56 Setup password tool builds default URL from settings (elastic/x-pack-elasticsearch#2146)
This change makes the setup password tool build the default URL from the settings provided by the
environment. This will ease the amount of work a user would have to do in order to run the tool as
http vs https will be selected automatically and the port/host will as well.

Original commit: elastic/x-pack-elasticsearch@79affe4a79
2017-08-01 14:13:08 -06:00
Ryan Ernst 45a55d16cf Bump master version to 7.0.0-alpha1 (elastic/x-pack-elasticsearch#2135)
This is the xpack side of
https://github.com/elastic/elasticsearch/pull/25876

Original commit: elastic/x-pack-elasticsearch@c86ea25009
2017-08-01 15:48:04 -04:00
Jay Modi ec11799003 Read the token passphrase earlier in the bootstrap check (elastic/x-pack-elasticsearch#2144)
This commit moves the reading of the token passphrase to the creation of the bootstrap check to
avoid issues with the secure settings keystore already being closed and thus causing issues during
startup.

Original commit: elastic/x-pack-elasticsearch@bba1cc832d
2017-08-01 13:04:34 -06:00
David Kyle 466d421abe [ML] Missing validations in analysis config (elastic/x-pack-elasticsearch#2103)
* Don’t set detector field names to empty strings
* Check summary count field and categorisation field names are not empty strings
* Check a detector has a by field when using multivariate by fields

Original commit: elastic/x-pack-elasticsearch@461be12f9f
2017-08-01 15:54:46 +01:00
David Roberts 8487e1e0fe [ML] Better handling of errors if native controller dies (elastic/x-pack-elasticsearch#2141)
If the native controller dies or is killed then requests to open jobs
now immediately return with an error that says what the problem is.
The error that is logged also now clearly records the problem.

Previously open job requests would time out if the native controller
was not running, and logged errors were not easy to understand without
in-depth knowledge of the ML code.

relates elastic/x-pack-elasticsearch#2140

Original commit: elastic/x-pack-elasticsearch@fc7f074d4a
2017-08-01 15:53:58 +01:00
Jay Modi 7291eb55fe Automatically enable AES 256 bit TLS ciphers when available (elastic/x-pack-elasticsearch#2137)
This commit adds detection of support for AES 256 bit ciphers and enables their use when the JVM
supports them. For OpenJDK, this is often the case without any changes but for the Oracle JVM, the
unlimited policy file needs to be installed. In order to simplify the work a user would need to do
we can detect this support and automatically enable the AES 256 bit versions of the ciphers we
already enable.

Original commit: elastic/x-pack-elasticsearch@5f23b18a1e
2017-08-01 07:36:35 -06:00
Alexander Reelsen 4bf5d9536a Tests: Remove @ClusterScope tests, create REST tests (elastic/x-pack-elasticsearch#2131)
Replacing integration tests with rest tests and unit tests, thus removing integration tests that require start of a new cluster. Removing unused testing methods

Original commit: elastic/x-pack-elasticsearch@265966d80c
2017-08-01 14:15:36 +02:00
Alexander Reelsen 547b0ebc1b X-Pack: Remove deprecated handlers, remove watcher restart action (elastic/x-pack-elasticsearch#2133)
The deprecated handlers should have been removed earlier, but are now
going to to away finally.

Also the watcher restart action has been removed, mainly because users
should not blindly restart, but always make sure, that watcher is
stopped correctly before restarting. This had been removed from the
transport action previously.

Original commit: elastic/x-pack-elasticsearch@78a5ec3c05
2017-08-01 14:14:51 +02:00
Alexander Reelsen 681ce6e2aa Watcher: Replace flaky WatchStatsTests with unit tests (elastic/x-pack-elasticsearch#2118)
This flaky tests was using sleep, latches and a custom script plugin,
causing issues with stopping/starting tests.

This removes the integration tests and replaces it with a unit test.

Also removed a couple of unused ctor/method parameters as cleanup.

relates elastic/x-pack-elasticsearch#1639

Original commit: elastic/x-pack-elasticsearch@2a42faf2db
2017-08-01 09:50:32 +02:00
Alexander Reelsen 9ea032998b Watcher: Remove REST hijacking operations (elastic/x-pack-elasticsearch#2132)
As all the triggering is now done on the shards, people can use
bulk, *-by-query and regular index/delete operations on the watcher
index and do not need to use the dedicated APIs.

We can now remove the long existing rest operation hijacking code.

Original commit: elastic/x-pack-elasticsearch@08f4f4c3de
2017-08-01 09:49:28 +02:00
Jay Modi 7fecca6329 Fix handling of exceptions for concurrent attempts to create security index (elastic/x-pack-elasticsearch#2120)
This commit fixes the handling of some exceptions when we attempt to create the security index and
alias. The issue here is provoked by a test that is currently muted with an AwaitsFix,
GroupMappingTests, which will be unmuted in another change.

Original commit: elastic/x-pack-elasticsearch@55f6b656cb
2017-07-31 12:34:14 -06:00
Luca Cavanna dac28fe4e0 Adapt to AbstractQueryBuilder upstream changes
AbstractQueryBuilder doesn't extend ToXContentToBytes anymore

Original commit: elastic/x-pack-elasticsearch@6c1b948ccb
2017-07-31 17:38:56 +02:00
Jason Tedor c51e6e1cd6 Remove unnecessary directory push/pop from scripts
This commit proposes removing an unnecessary directory push/pop from the
X-Pack scripts. It is not clear exactly why these were added, the
original change was almost three years ago in
elastic/x-pack@ea9ba7cdd0 but
unfortunately the commit message does not elucidate the exact the
problem, nor is there an associated pull request. This change has
propogated into all of the X-Pack scripts yet still the reasons are
unclear. The little that we can glean from the commit message is that
there was a problem with the default paths if the script was executed
outside of the Elasticsearch home. It seems that such issues have been
addressed by the recent introduction of elasticsearch-env so maybe we
can simplify these scripts here?

Relates elastic/x-pack-elasticsearch#2125

Original commit: elastic/x-pack-elasticsearch@9548c47743
2017-07-31 23:13:22 +09:00
Jason Tedor a80b1e4de1 Exit Windows scripts promptly on failure
When invoking the elasticsearch-env.bat or x-pack-env.bat batch scripts
on Windows, if these scripts exits due to an error (e.g., Java can not
be found, or the wrong version of Java is found), then the script
exits. Sadly, on Windows, this does not also terminate the caller,
instead returning control. This means we have to explicitly exit so that
is what we do in this commit.

Relates elastic/x-pack-elasticsearch#2126

Original commit: elastic/x-pack-elasticsearch@18645db62c
2017-07-31 20:39:59 +09:00
Jason Tedor 351febe031 Introduce elasticsearch-env for Windows
This commit refactors the batch scripts on Windows to use the
elasticsearch-env.bat script, and introduces a Windows version of
x-pack-env.bat for reuse in the scripts as well.

Relates elastic/x-pack-elasticsearch#2124

Original commit: elastic/x-pack-elasticsearch@faacb40dca
2017-07-30 09:31:52 +09:00
Chris Earle 45964561eb [Security] Add 'read_cross_cluster' privilege for .monitoring indices (elastic/x-pack-elasticsearch#2111)
This allows 6.0+ monitoring clusters to be used out of the box with CCS for extended read-only access.

Original commit: elastic/x-pack-elasticsearch@2b1e4ca4e4
2017-07-28 16:02:20 -04:00
Nik Everett 38e31cb4f6 Fix compiling sql after merge
Original commit: elastic/x-pack-elasticsearch@c0fc122e81
2017-07-28 12:39:08 -04:00
Jay Modi db4c00b565 Update the full cluster restart tests to be more generic (elastic/x-pack-elasticsearch#2107)
The full cluster restart tests are currently geared towards the 5.6 -> 6.0 upgrade and have some
issues when the versions are changed to 6.x -> 7.0. One issue is a real code issue in that the
security code always expects the mappings to have the same version as the version of the node, but
we no longer update the mappings on the security index during a rolling upgrade. We know look at
the index format to determine if the index is up to date.

Original commit: elastic/x-pack-elasticsearch@14c1c72ff6
2017-07-28 10:31:44 -06:00
Nik Everett b755b3e543 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@b80d0cba1d
2017-07-28 12:06:32 -04:00
David Kyle 694cfd6f0e Fix javadoc typo
Original commit: elastic/x-pack-elasticsearch@ef901af9cc
2017-07-28 16:24:24 +01:00
Lisa Cawley e7248c579c [DOCS] Fix licensing documentation links (elastic/x-pack-elasticsearch#2108)
Original commit: elastic/x-pack-elasticsearch@7b0f74348f
2017-07-28 08:19:55 -07:00
David Kyle f9104b7127 [ML] Lookback one extra bucket for histograms (elastic/x-pack-elasticsearch#2084)
Original commit: elastic/x-pack-elasticsearch@b9b4d3977f
2017-07-28 16:14:27 +01:00
Yannick Welsch 3a265de458 Revert "Mute IndexAuditTrailTests"
This reverts commit elastic/x-pack-elasticsearch@2e4a837dae.

Original commit: elastic/x-pack-elasticsearch@d8841cd2ce
2017-07-28 14:56:48 +02:00
Yannick Welsch 40174e4dcf [TEST] Set proper "transport.type" for transport client in IndexAuditTrailTests
Fixes issues seen on CI.

relates elastic/x-pack-elasticsearch#2115

Original commit: elastic/x-pack-elasticsearch@0094225a2e
2017-07-28 14:56:36 +02:00
David Kyle f99838e1bf Mute IndexAuditTrailTests
Original commit: elastic/x-pack-elasticsearch@2e4a837dae
2017-07-28 13:41:20 +01:00
David Kyle 21b437e140 Mute ShardsCollectorTests.testShardsCollectorMultipleIndices
Original commit: elastic/x-pack-elasticsearch@c2a6c05536
2017-07-28 11:42:51 +01:00
Martijn van Groningen 5f51dd813c XPack changes for: https://github.com/elastic/elasticsearch/pull/25456
Original commit: elastic/x-pack-elasticsearch@9d1d3c3fb4
2017-07-28 12:25:27 +02:00
Yannick Welsch 51e87bf290 Move tribe to a module (elastic/x-pack-elasticsearch#2088)
Companion PR to elastic/elasticsearch#25778

Original commit: elastic/x-pack-elasticsearch@a3355802e9
2017-07-28 11:23:52 +02:00
Jason Tedor 8176adfc26 Fix retrieve roles and users tests
This commit removes the use of a now removed --path.conf command-line
flag from the retrieve roles and users tests.

Original commit: elastic/x-pack-elasticsearch@30d5f5f648
2017-07-28 18:04:05 +09:00
Jason Tedor e77db8faf4 Fix users tool tests
This commit fixes the users tool command tests which were broken because
of a guard added that es.path.conf is set. We do not want to set this
system property in tests so instead we override createEnv where the
problematic guard exists.

Original commit: elastic/x-pack-elasticsearch@78b757695b
2017-07-28 17:32:39 +09:00
Jason Tedor d8277942ac Fix list X-Pack extension command tests
This commit fixes the list X-Pack extension command tests which were
broken because of a guard added that es.path.conf is set. We do not want
to set this system property in tests so instead we override createEnv
where the problematic guard exists.

Original commit: elastic/x-pack-elasticsearch@b1bc4ddcb8
2017-07-28 17:15:23 +09:00
Jason Tedor aade36eff3 Fix failing CLI tests
This commit fixes some failing CLI tests. The failure here is that a
guard against the system property es.path.conf was added yet these tests
were not adapted for this change. This commit implements this adapation
which overrides the createEnv method where the problematic guard is
invoked. We do this to avoid having to set es.path.conf in tests.

Original commit: elastic/x-pack-elasticsearch@20e1724823
2017-07-28 17:00:25 +09:00
Jason Tedor af7ec7a213 Pass config path as a system property
This commit responds to an upstream change which removes the --path.conf
command-line flag and instead uses the replacement mechanism for setting
the configuration path via the system property es.path.conf.

Relates elastic/x-pack-elasticsearch#2113

Original commit: elastic/x-pack-elasticsearch@4fefbffecb
2017-07-28 12:15:37 +09:00
Nik Everett ec36875872 Sql security take 1 (elastic/x-pack-elasticsearch#2106)
Add some basic security testing/integration.

The good news:
1. Basic security now works. Users without access to an index can't run sql queries against it. Without this change they could.
2. Document level security works! At least so far as I can tell.

The work left to do:
1. Field level security doesn't work properly. I mean, it kind of works in that the field's values don't leak but it just looks like they all have null values.
2. We will need to test scrolling.
3. I've only added tests for the rest sql action. I'll need to add tests for jdbc and the CLI as well.
4. I've only added tests for `SELECT` and have ignored stuff like `DESCRIBE` and `SHOW TABLES`.

Original commit: elastic/x-pack-elasticsearch@b9909bbda0
2017-07-27 17:40:14 -04:00
Jay Modi 3a1b64bb12 Create the cryptoservice later in startup process (elastic/x-pack-elasticsearch#2087)
This commit moves the creation of the CryptoService to the createComponents method so that bootstrap
checks have been checked before the crypto service is instantiated. The cryptoservice was changed to
expect that the bootstrap check has passed before being instantiated in elastic/x-pack-elasticsearch#1831.

Original commit: elastic/x-pack-elasticsearch@cf11cf4782
2017-07-27 14:03:05 -06:00
Nik Everett f2a4e29039 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@0d4ad7e19a
2017-07-27 09:06:56 -04:00
David Kyle b95d3f7bf4 Unmute test after fix
Original commit: elastic/x-pack-elasticsearch@cceb3e7237
2017-07-27 11:34:52 +01:00
Tim Brooks 1603823f2c Ensure that security client is used in x-pack
This commit is fixing an issue with the build. Currently we have a
scenario where a call to build a transport client is not using the
security client. This modifies the settings source to ensure that the
security client is used.

Original commit: elastic/x-pack-elasticsearch@2d6ea1f4e4
2017-07-26 13:36:20 -05:00
Luca Cavanna cb20de5d95 Adapt to removal of XContentHelper#toString(ToXContent) (elastic/x-pack-elasticsearch#2072)
Original commit: elastic/x-pack-elasticsearch@35f6ac23a1
2017-07-26 16:01:09 +02:00
Igor Motov 4da7ae21bc Persistent Tasks: remove unused isCurrentStatus method (elastic/x-pack-elasticsearch#2076)
Removes a method that is no longer used in production code.

Relates to elastic/x-pack-elasticsearch#957

Original commit: elastic/x-pack-elasticsearch@84fcb9db8a
2017-07-26 08:40:22 -04:00
Simon Willnauer 3914d66cea Followup for elastic/elasticsearch#25885
Original commit: elastic/x-pack-elasticsearch@0397e68efd
2017-07-26 09:17:40 +02:00
Igor Motov 2c8f96efc7 Sql should run in ci (elastic/x-pack-elasticsearch#2080)
Fixes several minor issue to enable sql branch to run in CI.

Original commit: elastic/x-pack-elasticsearch@b479cff92c
2017-07-25 14:59:50 -04:00
David Kyle 14c88ca15f Correct long line length
Original commit: elastic/x-pack-elasticsearch@e77f473b5b
2017-07-25 16:59:42 +01:00
David Kyle 74d06216c2 [ML] Accept more varied Datafeed Aggregations (elastic/x-pack-elasticsearch#2038)
Original commit: elastic/x-pack-elasticsearch@ec1477f41c
2017-07-25 16:45:47 +01:00
David Kyle 8f6d9df96e [ML] Check influencer names are valid (elastic/x-pack-elasticsearch#2073)
Original commit: elastic/x-pack-elasticsearch@75869cacb3
2017-07-25 10:41:51 +01:00
Tim Vernum 9ab6d3cbc3 [Security] Support PKCSelastic/x-pack-elasticsearch#12 keystores (elastic/x-pack-elasticsearch#2066)
Adds support for reading PKCSelastic/x-pack-elasticsearch#12 files as SSL keystores/truststores.

Original commit: elastic/x-pack-elasticsearch@1855ad6173
2017-07-25 17:31:37 +10:00
Nik Everett 39b9b55e19 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@1ada71a4f4
2017-07-24 15:04:59 -04:00
Michael Basnight 6a7e51d9c0 Use shaded rest client dependencies
This commit modifies all org.apache.http to use the shaded rest clients
org.elasticsearch.client.http packages. It also removes a few unused
licenses due to the change.

Relates elastic/elasticsearch#25780

Original commit: elastic/x-pack-elasticsearch@8605560232
2017-07-24 12:56:17 -05:00
Chris Earle b302ff32e8 [Monitoring] Remove BWC Layer for Marvel (2.3 - 2.4) in 6.0 (elastic/x-pack-elasticsearch#1990)
This removes all BWC code that assisted in reading Marvel indices in 6.0.

Original commit: elastic/x-pack-elasticsearch@253fbf9a73
2017-07-24 10:23:06 -04:00