Commit Graph

1206 Commits

Author SHA1 Message Date
jaymode 7b8d92a2a7 Test: fix AuthenticationService tests timeouts due to incorrect stream sizes
The AuthenticationService#testInvalidToken would cause a suite timeout in the case of an exception
due to a incorrect stream size as the latch was never counted down. This fixes the missing latch
countdown.

relates elastic/x-pack-elasticsearch#2615

Original commit: elastic/x-pack-elasticsearch@e838e6e912
2017-09-25 13:02:22 -06:00
Tim Brooks 6099660643 Update LicenseMetadata version check to be 6.1
This is related to elastic/x-pack-elasticsearch#1941. It modifies to the version check in
LicenseMetadata because this elastic/x-pack-elasticsearch#2419 has been backported to 6.x.

Original commit: elastic/x-pack-elasticsearch@052358b550
2017-09-25 09:42:37 -06:00
Tanguy Leroux 9bd2da86db [Tests] Reenable MonitoringIT
Checking the size of the map doesn't make sense when each filter is
checked independently right after.

Original commit: elastic/x-pack-elasticsearch@58e5d3401d
2017-09-25 16:01:18 +02:00
Simon Willnauer e7b5702f50 Adopt settings cleanups from core (elastic/x-pack-elasticsearch#2605)
Relates to elastic/elasticsearch#26739

Original commit: elastic/x-pack-elasticsearch@dd13d099de
2017-09-25 12:27:07 +02:00
Tanguy Leroux 1bb9c4fe71 [Tests] Mute MonitoringIT
see https://github.com/elastic/x-pack-elasticsearch/issues/2609

Original commit: elastic/x-pack-elasticsearch@e2e2393e62
2017-09-25 09:14:50 +02:00
Martijn van Groningen e93e7cf816 Revert "reset me"
This reverts commit elastic/x-pack-elasticsearch@3d44c14cd9.

Original commit: elastic/x-pack-elasticsearch@00203223b4
2017-09-25 08:56:01 +02:00
Martijn van Groningen ad3eb997c6 reset me
Original commit: elastic/x-pack-elasticsearch@3d44c14cd9
2017-09-25 08:52:51 +02:00
Jason Tedor b4f7d56c35 Whitelist global checkpoint sync actions
This commit whitelists the global checkpoint sync actions as otherwise
these actions do not have privileges to run as the system user and will
be denied as unauthorized.

Relates elastic/x-pack-elasticsearch#2604

Original commit: elastic/x-pack-elasticsearch@598ae1ff50
2017-09-22 16:27:51 -04:00
Tim Brooks 6b2e7fbed8 Add api to upgrade from basic to trial license (elastic/x-pack-elasticsearch#2419)
This is related to elastic/x-pack-elasticsearch#1941.

Currently we support self-generating either a basic or trial license at
cluster startup. With the addition of the basic option, it is possible
that a user would choose to self-generate and eventually register a
basic license.

This commit allows a user to upgrade to a 30-day trial license if they
have not already utilized this 30-day trial license before. Additionally
it adds a get route to check if the user is eligible to upgrade. This
route will allow kibana to implement a cleaner UI.

Original commit: elastic/x-pack-elasticsearch@7f19b33a08
2017-09-22 14:18:07 -06:00
Tanguy Leroux 6957282b50 [Monitoring] Fix MonitoringIT on Windows
Original commit: elastic/x-pack-elasticsearch@7a835c179d
2017-09-22 18:21:44 +02:00
Chris Earle 4cf8348209 [Monitoring] Remove Beats Support until wanted (elastic/x-pack-elasticsearch#2592)
This removes the creation and handling of the Beats monitoring template and its data until we actually expect to support it (most likely 6.2 - 6.3).

Original commit: elastic/x-pack-elasticsearch@2dc8abbb37
2017-09-22 11:12:44 -04:00
David Kyle ce77b076ad [ML] Adapt memory limit thresholds in test (elastic/x-pack-elasticsearch#2581)
* Adapt memory limit thresholds in test. Caused by elastic/machine-learning-cpp#305

Original commit: elastic/x-pack-elasticsearch@2984f35b2c
2017-09-21 12:52:31 +01:00
Albert Zaharovits f435145060 Fixes Javadoc generation for @SuppressForb adnot (elastic/x-pack-elasticsearch#2583)
Original commit: elastic/x-pack-elasticsearch@75ce6ac028
2017-09-21 14:28:57 +03:00
Albert Zaharovits a8ea4f2d90 Authz FieldExtractor support for TermInSet query (elastic/x-pack-elasticsearch#2539)
FieldExtractor for TermInSet query.

Closes: elastic/x-pack-elasticsearch#1104

Original commit: elastic/x-pack-elasticsearch@7a46361c86
2017-09-21 11:35:34 +03:00
Albert Zaharovits 6b51f5e6ca Update BouncyCastle version from 1.55 to 1.58 (elastic/x-pack-elasticsearch#2548)
Update BouncyCastle version from 1.55 to 1.58

Requires regeneration of .project and .classpath files, e.g.
gradle cleanEclipse & gradle eclipse

Closes: elastic/x-pack-elasticsearch#2332

Original commit: elastic/x-pack-elasticsearch@5806fd4204
2017-09-21 11:30:42 +03:00
Albert Zaharovits c84c48fa01 Usability improvement for the password bootstrap tool (reserved users) (elastic/x-pack-elasticsearch#2444)
Generate or prompt for passwords of the reserved users, validating strength
and typos, then issue change requests, keeping elastic user change request last.

Closes: elastic/x-pack-elasticsearch#2424

Original commit: elastic/x-pack-elasticsearch@1f827d393c
2017-09-21 11:26:28 +03:00
Tanguy Leroux 34bd72e69b [Monitoring] Remove monitoring resolvers (elastic/x-pack-elasticsearch#2566)
Now all monitoring documents are indexed in timestamped indices dedicated 
to each product, the old monitoring's resolvers and all the associated plumbing 
can be removed from the Elasticsearch monitoring plugin.

This pull request merges the feature branch feature/monitoring-summer-cleanup 
into master. All commits have been independently reviewed as part of elastic/x-pack-elasticsearch#2226:

* [Monitoring] Clean up monitoring doc (elastic/x-pack-elasticsearch#2208)
* [Monitoring] Remove ClusterStatsResolver (elastic/x-pack-elasticsearch#2209)
* [Monitoring] Remove IndexRecoveryResolver (elastic/x-pack-elasticsearch#2229)
* [Monitoring] Remove JobStatsResolver (elastic/x-pack-elasticsearch#2230)
* [Monitoring] Remove ShardsResolver (elastic/x-pack-elasticsearch#2233)
* [Monitoring] Fix missing instanceof in exporter bulks
* [Monitoring] Remove NodeStatsResolver (elastic/x-pack-elasticsearch#2244)
* [Monitoring] Remove IndexStatsResolver & IndicesStatsResolver  (elastic/x-pack-elasticsearch#2247)
* [Monitoring] Remove source node attributes and add timestamp (elastic/x-pack-elasticsearch#2285)
* [Monitoring] Remove last resolver (elastic/x-pack-elasticsearch#2330)
* [Monitoring] Make MonitoringDoc an abstract class (elastic/x-pack-elasticsearch#2451)
* [Monitoring] Fix TransportMonitoringBulkActionTests
* [Monitoring] Remove old resolver tests and add a single integration test (elastic/x-pack-elasticsearch#2453)

It does not introduce any new feature, so it could also go in 6.0.

relates elastic/x-pack-elasticsearch#2226


Original commit: elastic/x-pack-elasticsearch@e3d5cec28c
2017-09-21 09:36:03 +02:00
Tim Sullivan 723d894753 [Monitoring/Cluster Alerts] Fix the email message when cluster license expiration is resolved (elastic/x-pack-elasticsearch#2557)
* [Monitoring/Cluster Alerts] Fix the email message for cluster license expiration resolved

* fix making payload.message show only when new

Original commit: elastic/x-pack-elasticsearch@6d54b02913
2017-09-20 15:44:14 -07:00
Igor Motov 4964239f4c Tests: Remove AwaitsFix from TemplateUpgraderTests
I believe this problem should have been already fixed by elastic/elasticsearch#26698.

Original commit: elastic/x-pack-elasticsearch@0ec93f2803
2017-09-20 14:12:24 -04:00
Dimitrios Athanasiou a5e8589b38 [ML][TEST] Add integ test for elastic/x-pack-elasticsearch#2519
Relates elastic/x-pack-elasticsearch#2519

Original commit: elastic/x-pack-elasticsearch@91b38a62bb
2017-09-20 18:59:34 +01:00
Dimitris Athanasiou fad98d784f [ML] Align aggregated data extraction to histogram interval (elastic/x-pack-elasticsearch#2553)
When the datafeed uses aggregations and in order to accommodate
derivatives, an extra bucket is queried at the beginning of each
search. In order to avoid visiting the same bucket twice, we need
to search buckets aligned to the histogram interval. This allows
us to steer away from partial buckets, and thus avoid the problem
of dropping or duplicating data.

relates elastic/x-pack-elasticsearch#2519

Original commit: elastic/x-pack-elasticsearch@e03dde5fea
2017-09-20 16:45:06 +01:00
Hendrik Muhs 3c517902f2 re-enable categorization IT (elastic/x-pack-elasticsearch#2534)
re-enables categorization IT after fixing https://github.com/elastic/machine-learning-cpp/issues/279 upstream

Original commit: elastic/x-pack-elasticsearch@398a668125
2017-09-20 11:39:37 +02:00
Yannick Welsch 8648153f0e Deguice ActionFilter (elastic/x-pack-elasticsearch#2533)
Companion PR to elastic/elasticsearch#26691

Original commit: elastic/x-pack-elasticsearch@3fceb54809
2017-09-20 10:30:23 +02:00
Tal Levy 8b1021ccad blacklist two license rest tests when build.snapshot=false (elastic/x-pack-elasticsearch#2559)
There are two rest tests that rely on hardcoded license
signatures that use the dev public key. These tests fail
when tests are run with build.snapshot=false. This Commit
blacklists these two tests in that scenario

relates elastic/x-pack-elasticsearch#2527.

Original commit: elastic/x-pack-elasticsearch@7581e8d699
2017-09-19 16:56:11 -07:00
Nik Everett ea66433899 Mark periodically failing test as AwaitsFix
This should reduce our build failure emails. The failure is already
tracked at https://github.com/elastic/x-pack-elasticsearch/issues/2421.
It isn't actively being worked but will be soon.

Original commit: elastic/x-pack-elasticsearch@6baf55de42
2017-09-19 09:59:26 -04:00
David Roberts 67055877a5 [ML] Avoid creating spurious 0 "actual" values for model plot documents (elastic/x-pack-elasticsearch#2535)
Some model plot documents should not have an "actual" value, for example
when no input events were seen for a meean/min/max detector in a particular
bucket.  Prior to this change we would set the "actual" value to 0 for such
model plot documents.  Following this change no "actual" value will be
present in these documents.

Only newly created model plot documents are affected.  Model plot documents
that were incorrectly written in the past will remain wrong forever.

relates elastic/x-pack-elasticsearch#2528

Original commit: elastic/x-pack-elasticsearch@47a7365f59
2017-09-19 10:30:43 +01:00
Tim Vernum aec2308228 Allow AD realm to perform 'run-as' lookups (elastic/x-pack-elasticsearch#2531)
- Marks the AD Session factory as supporting "lookup" (Refer: elastic/x-pack-elasticsearch@40b07b3)
- Adds "pool.enabled" as a registered setting on AD realm (Refer: elastic/x-pack-elasticsearch@40b07b3)
- Fixes LDAP user lookup that has been broken since 6.x (Refer: elastic/x-pack-elasticsearch@f796949)

Original commit: elastic/x-pack-elasticsearch@62ff6129a1
2017-09-19 14:50:26 +10:00
Tal Levy 5a090c14c1 convert more request objects to writeable (elastic/x-pack-elasticsearch#2457)
* convert more to writeable

* migrate streamable tests to writeable tests

Original commit: elastic/x-pack-elasticsearch@56794e5760
2017-09-18 13:20:02 -07:00
Suyog Rao 3a9aad5ece [Logstash] Remove version field from config mgmt
relates elastic/x-pack-elasticsearch#2405

Original commit: elastic/x-pack-elasticsearch@5bfd1b7a6d
2017-09-18 12:35:21 -07:00
Simon Willnauer 47214426e9 Use InputStreamStreamInputs validation to limit size of tokens (elastic/x-pack-elasticsearch#2537)
Relates to elastic/elasticsearch#26692
relates elastic/x-pack-elasticsearch#2493

Original commit: elastic/x-pack-elasticsearch@8e23868743
2017-09-18 19:25:08 +02:00
Dimitris Athanasiou 9b25d0edf7 [ML] Correctly ignore extra aggregation buckets (elastic/x-pack-elasticsearch#2530)
The problem here is that the code was ignoring buckets
whose start time was before the start time of the extractor.
However, this is not a good enough condition. For example,
when there are no data in the bucket extra bucket that is
being queried, the first bucket will be the one containing
the start time.

This commit fixes the issue by changing the condition to
ignore buckets before the first bucket that includes the
start time of the extraction.

relates elastic/x-pack-elasticsearch#2519 

Original commit: elastic/x-pack-elasticsearch@15c7d2655f
2017-09-18 12:10:19 +01:00
Tim Vernum dea82a07a2 Infer KeyStore type from pathname (elastic/x-pack-elasticsearch#2514)
If the keystore type is not explicitly specified, infer it from the filename.
Treats .p12, .pfx and .pkcs12 as being PKCS12, all others as jks.

This will allow certgen to produce PKCSelastic/x-pack-elasticsearch#12 files by default and make it easy to use them as x-pack keystores

Original commit: elastic/x-pack-elasticsearch@fc361f0d87
2017-09-18 14:21:19 +10:00
Chris Earle 24c2c62ca2 [Monitoring] Ignore .marvel* indices (elastic/x-pack-elasticsearch#2520)
Beginning with 7.0, the cleaner service will no longer automatically cleanup .marvel indices regardless of their age.

Original commit: elastic/x-pack-elasticsearch@5b90e6f62a
2017-09-15 13:35:45 -07:00
Tim Brooks b3914afd30 Reenable TribeWithSecurityIT tests (elastic/x-pack-elasticsearch#2511)
This is related to elastic/x-pack-elasticsearch#1996. These tests were disabled during the bootstrap
password work. They can now be reenabled. Additionally, I made the test
password used in tests consistent.

Original commit: elastic/x-pack-elasticsearch@5b490c8231
2017-09-15 12:50:54 -06:00
Simon Willnauer 96e01dce47 Only require TLS for standard/gold/platinum licenses elastic/x-pack-elasticsearch#2517
relates elastic/x-pack-elasticsearch#2463

Original commit: elastic/x-pack-elasticsearch@5213cf24f1
2017-09-15 20:21:15 +02:00
Chris Earle 9c9da2e1e4 [Monitoring] Remove Legacy Monitoring Indices (elastic/x-pack-elasticsearch#2513)
This changes Monitoring's Cleaner Service to remove any legacy Monitoring index that is appropriately old.

This includes any `.marvel-*` index and also the "data" indices used by both Marvel and 5.0 - 5.4 versions of X-Pack monitoring, as well as the legacy alerts index.

Original commit: elastic/x-pack-elasticsearch@8d99f5518b
2017-09-15 08:46:10 -07:00
jaymode 84dd719ab9 remove outdated comment
Original commit: elastic/x-pack-elasticsearch@06a51abb65
2017-09-15 09:03:36 -06:00
jaymode 344603e40f update text in TLSLicenseBootstrapCheck
Original commit: elastic/x-pack-elasticsearch@4ee6827566
2017-09-15 08:56:34 -06:00
Simon Willnauer 4d20586b24 [TEST] add integration test that ensures we reject license upgrades if TLS is not enabled
Original commit: elastic/x-pack-elasticsearch@dfbadb5e5f
2017-09-15 14:47:28 +02:00
Simon Willnauer c3066d1a51 Merge branch 'master' into tls_6.0
Original commit: elastic/x-pack-elasticsearch@9ce33bc7c3
2017-09-15 09:51:16 +02:00
Simon Willnauer 023bdb72b2 Add common-analysis plugin to several xpack integ tests (elastic/x-pack-elasticsearch#2501)
Several tests miss the common-analysis plugin in the old-style integ
tests causing odd exceptions in the test logs. This adds the missing plugin reference.

relates elastic/x-pack-elasticsearch#2363

Original commit: elastic/x-pack-elasticsearch@e4e6735408
2017-09-15 09:25:36 +02:00
jaymode 8997792875 Test: use TLS for plugin integ tests
Original commit: elastic/x-pack-elasticsearch@99971d7256
2017-09-14 15:57:28 -06:00
Michael Basnight fa0b854fb6 Update rest-api-spec to use bad_request (elastic/x-pack-elasticsearch#2507)
ref #elastic/elasticsearch#26539

Original commit: elastic/x-pack-elasticsearch@8b79a0769a
2017-09-14 15:59:29 -05:00
Andy Bristol 279c7e14fd [TEST] fix security template version check in rest tests (elastic/x-pack-elasticsearch#2506)
Since the template upgrade service was added, upgrades should
be performed by a node with the highest version in the cluster,
which may not be the master node.

Original commit: elastic/x-pack-elasticsearch@d66145de54
2017-09-14 12:16:20 -07:00
Jay Modi 57de66476c Disable TLS by default (elastic/x-pack-elasticsearch#2481)
This commit adds back the ability to disable TLS on the transport layer and also disables TLS by
default to restore the 5.x behavior. The auto generation of key/cert and bundled CA certificate
have also been removed.

Relates elastic/x-pack-elasticsearch#2463

Original commit: elastic/x-pack-elasticsearch@abc66ec67d
2017-09-14 12:18:54 -06:00
Simon Willnauer 1e14e14571 Prevent licenses to be upgraded to production unless TLS is configured (elastic/x-pack-elasticsearch#2502)
if a user tries to upgrade a license to a production license and has security
enabled we prevent the upgrade unless TLS is setup. This is a requirement now
if a cluster with security is running in prodcution.

Relates to elastic/x-pack-elasticsearch#2463

Original commit: elastic/x-pack-elasticsearch@d61ef3bcb1
2017-09-14 20:14:27 +02:00
Jay Modi 8d8baffe24 Add specific client and user for security index access (elastic/x-pack-elasticsearch#2492)
This change removes security index access from the xpack user by creating its own specific role
and adds a xpack security user that maintains the superuser role so that it can perform all
operations necessary for security.

Original commit: elastic/x-pack-elasticsearch@ad906bc913
2017-09-14 08:09:14 -06:00
David Roberts 104a3a323f [BUILD] Make AWS error message more informative
Original commit: elastic/x-pack-elasticsearch@42cca7ed82
2017-09-14 14:46:48 +01:00
Simon Willnauer 91b57ee63f Add bootstrap check that enforces TLS if a production license is in the local clusterstate (elastic/x-pack-elasticsearch#2499)
This change will enforce transport SSL to be enforced if security is enabled and the
license in the clusterstate is a production license. The cluster state is loaded from
local storage such that we don't need to join a cluster to make these checks. Yet, the cluster
might have already got a different license if the node got disconnected while the license got
downgraded and then TLS got disabled. This corner case requires manual intervention which
we consider ok given the simplicity of this change.

Relates to elastic/x-pack-elasticsearch#2463

Original commit: elastic/x-pack-elasticsearch@5765b7cd21
2017-09-14 13:52:53 +02:00
Hendrik Muhs 7d19264363 [ML-FC] Branch landing feature/ml (elastic/x-pack-elasticsearch#2500)
integrate forecasting feature branch into master

    - add endpoint xpack/ml/job/forecast to request forecasting on data of ml-jobs
       - current parameters: end time
    - persists forecast results into shared or own index
       - different runs are separated by a 'forecast id'

relates elastic/x-pack-elasticsearch#1838

Original commit: elastic/x-pack-elasticsearch@f9d701a6bc
2017-09-14 12:31:20 +02:00