Commit Graph

4922 Commits

Author SHA1 Message Date
David Roberts ee9c691858 Switch to tracking Elasticsearch/Kibana 5.3.0-SNAPSHOT (elastic/elasticsearch#651)
Original commit: elastic/x-pack-elasticsearch@75c54427ac
2017-01-06 10:06:08 +00:00
Martijn van Groningen 66a2f999e5 scheduler: prevent stopping scheduler before the scheduler was actually started
The start scheduler waits until the scheduler state has been set to started before returning.
Before this change after the scheduler state has been set to started, the scheduler would link itself to the task the start scheduler api has created.
If the stop scheduler api was called immediately after the start scheduler api then this could lead the stop scheduler api cancelling the task without
stopping the scheduler, as the scheduler could not have been linked to the task.

Now the scheduler gets linked to the task before the scheduler state is set to started, fixing the problematic situation discribed above.

Original commit: elastic/x-pack-elasticsearch@8334ae1967
2017-01-05 21:10:23 +01:00
Tanguy Leroux 19cbab4ac3 [Watcher] Update triggering interval in Jira integration tests
In Jira integration tests, some watches are triggered every second whereas they are executed using the watch execute API. This commit increases the triggering interval to 1d so that the watches are not executed on slow machines.

Original commit: elastic/x-pack-elasticsearch@4d0462bc00
2017-01-05 21:00:53 +01:00
javanna 80dc895450 remove ParseFieldMatcher usages from Script parsing code
Original commit: elastic/x-pack-elasticsearch@4be5f83084
2017-01-05 19:33:15 +01:00
Tim B f71733ec4d Replace com.sun.net.httpserver.Http/Https server usages with MockHttpServer. (elastic/elasticsearch#4476)
Original commit: elastic/x-pack-elasticsearch@4b2d184f53
2017-01-05 11:29:31 -06:00
Tanguy Leroux b72dd8a2d1 [Monitoring] Schedule data collection instead of sleeping (elastic/elasticsearch#4266)
This commit renames the current AgentService into MonitoringService and changes the way it works: it was previously based on thread sleeping and it now use thread scheduling instead. 

At every given time interval, a MonitoringExecution is executed. It first checks if monitoring data can be collected and if so it will collect data from Collectors and then export the data using the Exporters. There are cases where the data cannot be collected: when the service is stopping, when the interval has been set to -1 after the MonitoringExecution has been scheduled, or when the previous data collection is not yet terminated. In this last case, MonitoringExecution will still be executed at the given interval but will not collect any data.

All tasks are executed on the generic thread pool.

closes elastic/elasticsearch#2866

Original commit: elastic/x-pack-elasticsearch@d37b4d3731
2017-01-05 17:45:17 +01:00
Colin Goodheart-Smithe b9205142a4 fix checkstyle
Original commit: elastic/x-pack-elasticsearch@b86e94228b
2017-01-05 16:29:18 +00:00
Colin Goodheart-Smithe 8fdeedf61d fix generics compile error in Eclipse
Eclipse compiler needs a hint to tell it the generics for the comparator function

Original commit: elastic/x-pack-elasticsearch@b3d71d3465
2017-01-05 16:23:24 +00:00
David Kyle 70aa238327 Merge branch 'master' of github.com:elastic/prelert-legacy
Original commit: elastic/x-pack-elasticsearch@88e78a3417
2017-01-05 14:43:37 +00:00
David Kyle 42898ba421 Fix noisy log message about missing mappings
From the scheduler integration test

Original commit: elastic/x-pack-elasticsearch@02d3f58d83
2017-01-05 14:43:29 +00:00
Tanguy Leroux 3d2d1d49b6 [Watcher] Move Jira integration tests to smoke-test-watcher (elastic/elasticsearch#4534)
This commit moves the Jira rest integration tests from the smoke-test-watcher-with-mustache project to the smoke-test-watcher project.

Original commit: elastic/x-pack-elasticsearch@c6b03d557f
2017-01-05 15:22:59 +01:00
Alexander Reelsen 63f4bbba98 Watcher: Use Apache HttpClient for internal Watcher HttpClient (elastic/elasticsearch#4434)
Watcher: Use Apache HttpClient for internal Watcher HttpClient

The current implementation based on URLConnection has several drawbacks.

* If server returned HTTP header but then got stuck, no timeout would help, the connection remained stuck
* GET requests with a body were not supported, the method was silently changed to POST
* More complex handling of input/error stream handling, the body could not be read from a single input stream

NOTE: This is a BWC breaker. From now on every part of the URL needs to be encoded properly before it is configured in the requeust builder. This requires an upgrade of all watches.

Closes elastic/elasticsearch#1141

Original commit: elastic/x-pack-elasticsearch@bbc8f85dd8
2017-01-05 14:25:58 +01:00
David Roberts 2746edf21b Properly assert that large state documents can be consumed quickly
AssertBusy was not the correct way to do this

Original commit: elastic/x-pack-elasticsearch@c6535e7545
2017-01-05 13:07:56 +00:00
Martijn van Groningen 41f2f51df6 Made client calls in get buckets code non blocking.
Also merged the JobProvider#getBucket(...) method into Jobprovider#getBuckets(...) method, because
it contained a lot of similar logic, otherwise it had to be converted to use non blocking client calls too.

Part of elastic/elasticsearch#127

Original commit: elastic/x-pack-elasticsearch@b1e66b62cb
2017-01-05 13:58:46 +01:00
Tanguy Leroux 125a2c9c03 [Watcher] Enable Jira integration tests (elastic/elasticsearch#4331)
This commit enables the Jira integration tests with the Jira project and account provided by Edward Sy.

closes elastic/infraelastic/elasticsearch#1498

Original commit: elastic/x-pack-elasticsearch@78d1005064
2017-01-05 13:44:04 +01:00
Martijn van Groningen 51e1199860 Install prelert metadata and create required indices only once.
Original commit: elastic/x-pack-elasticsearch@12c8ba0ce0
2017-01-05 13:38:48 +01:00
David Roberts 27c9f39bf5 Speed up state processing (elastic/elasticsearch#642)
There was an N-squared algorithm in the state processing code, leading
to large state persistence eventually timing out.  Large state documents
are read from the network in 8KB chunks, and the old code was checking
ALL previously read chunks for separators every time a new chunk was read.

Fixes elastic/elasticsearch#635

Original commit: elastic/x-pack-elasticsearch@c814858c2c
2017-01-05 12:37:11 +00:00
Alexander Reelsen 9f43a5320e Watcher: Fix test failure in case watch alias was created before
An integration test was failing due to the possibility of the
.watch index being an alias (which only happens rarely).

Original commit: elastic/x-pack-elasticsearch@d53a4ebdd4
2017-01-05 11:44:41 +01:00
Nik Everett 93b16f0615 Fix another qa project
Original commit: elastic/x-pack-elasticsearch@9609036dde
2017-01-05 10:56:09 +01:00
Nik Everett f2ae490b32 Switch from standalone-test to standalone-rest-test
standalone-rest-test doesn't configure unit tests and for these
integTest only projects that is what we want.

Original commit: elastic/x-pack-elasticsearch@f576dfdfbb
2017-01-05 10:56:09 +01:00
Nik Everett 5b6bfffa9a Require either BuildPlugin or StandaloneTestBasePlugin to use RestTestPlugin
It used to be that RestTestPlugin "came with" StandaloneTestBasePlugin
but we'd like to use it with BuildPlugin for the high level rest client.

Also fix some license headers.

Original commit: elastic/x-pack-elasticsearch@3d5549d170
2017-01-05 10:56:09 +01:00
Alexander Reelsen cb7f916485 Watcher: Remove async code in sync code block (elastic/elasticsearch#4506)
Watcher: Remove async code in sync code block

When removing the watch store this code snippet sneaked in.
A call to get a watch in order to find out if it exists was async, but
the code checking the result was called immediately afterwards without
waiting for the result, thus always using the default value.

This also removes some unused code in TriggeredWatchStore.

Original commit: elastic/x-pack-elasticsearch@c47e70bf8a
2017-01-05 10:24:17 +01:00
David Roberts c7cfa56aaf Rename prelert to ml in endpoint base path (elastic/elasticsearch#639)
Original commit: elastic/x-pack-elasticsearch@ff6745e545
2017-01-05 08:18:43 +00:00
Tim Vernum c64ad22579 Use Version.compareTo when calculating minimum version (elastic/elasticsearch#4505)
Results from elasticsearch change elastic/elasticsearch#22378 (6ad5486)

Original commit: elastic/x-pack-elasticsearch@73c09657bb
2017-01-05 17:43:10 +11:00
Tim Vernum ca3bbc7da0 [TESTS] Test resolving ldap groups with preloaded attributes (elastic/elasticsearch#4530)
Add a test for the `UserAttributeGroupsResolver` when using attributes provided as a parameter.

(ported from commit elastic/x-pack@ebc62b7, elastic/elasticsearch#4522)

Original commit: elastic/x-pack-elasticsearch@6502fdb801
2017-01-05 17:42:28 +11:00
polyfractal f98b4a967c s/nocommit/norelease
Original commit: elastic/x-pack-elasticsearch@6523f5caa1
2017-01-04 16:50:51 -05:00
Zachary Tong 23aef2679d DELETING job status cluster state changes (elastic/elasticsearch#612)
Deleting a job now starts a three-step process:

1. Job status updated to DELETING
2. Physical index is deleted
3. Job removed from cluster state

When jobs are in DELETING, they cannot be modified/updated/changed at all.  Only jobs that are DELETING can actually be removed from the CS.

Original commit: elastic/x-pack-elasticsearch@2cd99a240c
2017-01-04 16:43:24 -05:00
Chris Earle 3c68e6dd23 Ignore 5m and 15m system load averages in resolver tests so that Mac users can run them
Original commit: elastic/x-pack-elasticsearch@4a2a75b046
2017-01-04 11:59:05 -05:00
Martijn van Groningen 090c7e792a in case of general exception no need to execute the rest of the method
Original commit: elastic/x-pack-elasticsearch@d0cdaf037d
2017-01-04 15:52:17 +01:00
Martijn van Groningen 5371cc8454 When failures happen during the scheduler's loopback run then make sure we either continue or stop.
Relates to elastic/elasticsearch#628

Original commit: elastic/x-pack-elasticsearch@c5726bd8e4
2017-01-04 15:52:17 +01:00
David Roberts a816ee27bb Fix error message for state index creation
Original commit: elastic/x-pack-elasticsearch@cd36db071e
2017-01-04 13:31:03 +00:00
Chris Earle 35b405d029 [Monitoring] Add 'logstash' type to old .monitoring-data-2 indices (elastic/elasticsearch#4517)
Add `logstash`, in addition to `kibana` type mappings if they are missing from the _existing_ .monitoring-data-2 indices. This allows existing indices to be upgraded in place to support the Logstash monitoring feature added in 5.2 without having to delete the .monitoring-data-2 index.

Original commit: elastic/x-pack-elasticsearch@864343a116
2017-01-04 00:43:30 -05:00
Thomas Neirynck b799c1603b Merge pull request elastic/elasticsearch#3480 from thomasneirynck/enhancement-tileurl
Add license id when retrieving map metadata

Original commit: elastic/x-pack-elasticsearch@ff50a72fb4
2017-01-03 19:15:09 -05:00
Chris Earle 0d62207f8f [Monitoring] BWC Allow New Types to be added to .monitoring-data-2 index (elastic/elasticsearch#4504)
* [Monitoring] BWC Allow New Types to be added to .monitoring-data-2 index

This adds the new Logstash (and technically Kibana) types to the index mapping if they're not already there. Pre-existing indices will be blocked from creating new types in the index due to the index setting. The index setting cannot be flipped without opening/closing the index, so manually adding the new types is easier.

Original commit: elastic/x-pack-elasticsearch@e85e800335
2017-01-03 16:47:09 -05:00
javanna cd2e608ecc Remove unused ParseFieldMatcher argument from SearchInput and SearchTransform parse methods
Original commit: elastic/x-pack-elasticsearch@6cf7323961
2017-01-03 22:39:48 +01:00
Martijn van Groningen 567153d73a test: create airline-data index as part of test setup
Original commit: elastic/x-pack-elasticsearch@81dea8b071
2017-01-03 22:35:31 +01:00
javanna 4359a0508f remove unused ParseFieldMatcher imports
Original commit: elastic/x-pack-elasticsearch@c5c4834a8b
2017-01-03 22:26:17 +01:00
David Roberts 852eeb106d Remove obsolete test files
Original commit: elastic/x-pack-elasticsearch@d708431007
2017-01-03 17:12:14 +00:00
Jay Modi e41b53c344 roles with FLS and/or DLS are ignored when unlicensed (elastic/elasticsearch#4481)
Currently, roles making use of field or document level security are still applied when
the license level does not enable field and document level security. There is no indication
that these roles are not being applied so it is misleading to users. This change prevents
these roles for applying to authorization and also adds a transient metadata to the response
that indicates which features of a role is unlicensed.

Additionally, this PR prevents the addition or modification of roles to include field or
document level security.

Closes elastic/elasticsearch#2472

Original commit: elastic/x-pack-elasticsearch@c9455958f5
2017-01-03 12:06:33 -05:00
Chris Earle f4b9e794e8 [Monitoring] Fix test that fails with Cgroups missing on some machines
Original commit: elastic/x-pack-elasticsearch@23b7c2a25a
2017-01-03 12:02:12 -05:00
David Kyle f6564c03e2 Add Influencer.influencer_field_value to custom all field (elastic/elasticsearch#624)
Original commit: elastic/x-pack-elasticsearch@5aa98d84dc
2017-01-03 16:46:19 +00:00
David Kyle 7a7ec570ae Flushing a closed job should not return a 200 (elastic/elasticsearch#625)
Original commit: elastic/x-pack-elasticsearch@419389e1af
2017-01-03 16:44:13 +00:00
David Kyle c58d1d43d6 Null check for anomaly score filter values and default the page params
Fixes broken tests

Original commit: elastic/x-pack-elasticsearch@2ebc42515a
2017-01-03 15:33:19 +00:00
David Kyle f8309a0706 Fix validating params for single bucket requests (elastic/elasticsearch#622)
Original commit: elastic/x-pack-elasticsearch@42f8771bf8
2017-01-03 14:59:01 +00:00
javanna 33653a8865 Remove ParseFieldMatcher usage from SearchRequest
Original commit: elastic/x-pack-elasticsearch@e1b08ab801
2017-01-03 14:48:13 +01:00
Daniel Mitterdorfer dd26c93f68 Eliminate unneccessary declaration of IOException
Original commit: elastic/x-pack-elasticsearch@9f71ff4b55
2017-01-03 12:41:06 +01:00
Martijn van Groningen db3358166b made checkstyle happy
Original commit: elastic/x-pack-elasticsearch@6e12c81c33
2017-01-03 12:38:12 +01:00
Martijn van Groningen 134ce2f074 test: added a temporary workaround in on check that happens after a suite has completed,
with the fix we also make sure that prelert metatadata is taken into account when verifying the cluste state consistency

Original commit: elastic/x-pack-elasticsearch@1deaec3836
2017-01-03 12:23:23 +01:00
Martijn van Groningen 16bd07b603 register prelert metadata as named writables
Original commit: elastic/x-pack-elasticsearch@628a4aa154
2017-01-03 11:59:37 +01:00
David Roberts 161eefe7d8 Remove PROTO constants
This is a follow-up to the previous commit

Original commit: elastic/x-pack-elasticsearch@4d1e52fe43
2017-01-03 10:49:00 +00:00