Commit Graph

5695 Commits

Author SHA1 Message Date
Dimitris Athanasiou 3e9c36838d [ML] Remove get model snapshot by description functionality (elastic/x-pack-elasticsearch#1288)
relates elastic/x-pack-elasticsearch#1284

Original commit: elastic/x-pack-elasticsearch@780feea5c1
2017-05-03 13:20:52 +01:00
Luca Cavanna 796e23a02a Adapt to ActionRequestBuilder#execute changes (elastic/x-pack-elasticsearch#1263)
Original commit: elastic/x-pack-elasticsearch@c2d7aa562c
2017-05-03 11:21:06 +02:00
Hendrik Muhs 9d8fea9a32 [ML] expose low and high median functions (elastic/x-pack-elasticsearch#1272)
Expose low and high median functions implemented downstream in machinelearning-cpp.

Original commit: elastic/x-pack-elasticsearch@3863e9d41c
2017-05-03 09:46:10 +02:00
Daniel Mitterdorfer 240bd5bfa7 Mute EllipticCurveSSLTests
Relates elastic/x-pack-elasticsearch#1278

Original commit: elastic/x-pack-elasticsearch@3e5dbf5fe6
2017-05-03 07:57:27 +02:00
Chee Wee 1e0caee90d Corrected a mistake on disabling a user (elastic/x-pack-elasticsearch#1135)
Corrected a mistake on disabling a user, when it should be enabling.

Please review the changes.

Original commit: elastic/x-pack-elasticsearch@352997e427
2017-05-03 10:55:26 +08:00
Lisa Cawley 33c50f1201 [DOCS] Add property table for ML Update Jobs API (elastic/x-pack-elasticsearch#1268)
* [DOCS] Add property table for ML Update Jobs API

* [DOCS] Updates based on feedback for ML Update Jobs API

* [DOCS] Removed detector properties from ML Update Jobs API

* [DOCS] Fixes typos

Original commit: elastic/x-pack-elasticsearch@68d1b5598c
2017-05-02 15:38:41 -07:00
Lisa Cawley 9b2fb6ac16 [DOCS] Change "data feed" to "datafeed" in Machine Learning documentation (elastic/x-pack-elasticsearch#1277)
* [DOCS] Add xpackml attribute to XPack Reference

* [DOCS] Use attribute for datafeed terms

Original commit: elastic/x-pack-elasticsearch@f37bf48ee4
2017-05-02 12:45:42 -07:00
Alexander Reelsen 3a0bc504a9 Watcher: Exeuting a watch returns a proper 404 (elastic/x-pack-elasticsearch#1273)
A test failure uncovered that handling non existing watches in the
execute watch API could lead to 500 errors instead of 404 because
they were not handled correctly.

relates elastic/x-pack-elasticsearch#1120

Original commit: elastic/x-pack-elasticsearch@c17a07ae3e
2017-05-02 16:01:45 +02:00
Alexander Reelsen ff6283bf80 Tests: Ignore existing watches index in REST test
This test should ensure, that a missing watch on an existing index
causes an error, thus it can ignore if the .watches index already
exists.

relates elastic/x-pack-elasticsearch#1155

Original commit: elastic/x-pack-elasticsearch@89c56cb2be
2017-05-02 14:16:19 +02:00
Alexander Reelsen 50241717d3 Tests: Use execute watch API instead of persisting watch
To speed up the test and ensure no other failures occur,
simply use the execute watch API instead of persisting the watch
and executing then.

relates elastic/x-pack-elasticsearch#1350

Original commit: elastic/x-pack-elasticsearch@2888fb0e93
2017-05-02 14:04:04 +02:00
Tanguy Leroux 9e14141a8e [Test] Fix NPE in LocalExporterTests
This commit fixes a NullPointerException in the LocalExporterTests.

Original commit: elastic/x-pack-elasticsearch@e28516135f
2017-05-02 13:55:47 +02:00
David Kyle f3f387bd9c [ML] Give jobs meaningful and unique names in the yaml rest tests. (elastic/x-pack-elasticsearch#1271)
Original commit: elastic/x-pack-elasticsearch@b096de55c1
2017-05-02 11:36:33 +01:00
Alexander Reelsen c62f6f8177 Watcher: Distributed watch execution (elastic/x-pack-elasticsearch#544)
The distribution of watches now happens on the node which holds the
watches index, instead of on the master node. This requires several
changes to the current implementation.

1. Running on shards and replicas
   In order to run watches on the nodes with the watches index on its
   primaries and replicas. To ensure that watches do not run twice, there is
   a logic which checks the local shards, runs a murmurhash on the id and
   runs modulo against the number of shards and replicas, this is the way to
   find out, if a watch should run local. Reloading happens
2. Several master node actions moved to a HandledTransportAction, as they
   are basically just aliases for indexing actions, among them the
   put/delete/get watch actions, the acknowledgement action, the de/activate
   actions
3. Stats action moved to a broadcast node action, because we potentially
   have to query every node to get watcher statistics
4. Starting/Stopping watcher now is a master node action, which updates
   the cluster state and then listeners acts on those. Because of this watches
   can be running on two systems, if you those have different cluster state
   versions, until the new watcher state is propagated
5. Watcher is started on all nodes now. With the exception of the ticker
   schedule engine most classes do not need a lot of resources while running.
   However they have to run, because of the execute watch API, which can hit
   any node - it does not make sense to find the right shard for this watch
   and only then execute (as this also has to work with a watch, that has not
   been stored before)
6. By using a indexing operation listener, each storing of a watch now
   parses the watch first and only stores on successful parsing
7. Execute watch API now uses the watcher threadpool for execution
8. Getting the number of watches for the stats now simply queries the
   different execution engines, how many watches are scheduled, so this is
   not doing a search anymore

There will be follow up commits on this one, mainly to ensure BWC compatibility.

Original commit: elastic/x-pack-elasticsearch@0adb46e658
2017-05-02 10:12:46 +02:00
debadair dae762913d [DOCS] Added xpack.monitoring.ui.enabled to monitoring settings.
Original commit: elastic/x-pack-elasticsearch@3dd136ed57
2017-05-01 17:18:51 -07:00
Jason Tedor 718518fe85 Disable setting available processors in tests
Within the same JVM, setting the number of processors available to Netty
can only be done once. However, tests randomize the number of processors
and so without intervention would attempt to set this value multiple
times. Therefore, we need to use a flag that prevents setting this value
in tests.

Relates elastic/x-pack-elasticsearch#1266

Original commit: elastic/x-pack-elasticsearch@d127149725
2017-05-01 19:27:45 -04:00
Lisa Cawley a615532866 [DOCS] ML 5.4 docs final tuning (elastic/x-pack-elasticsearch#1265)
Original commit: elastic/x-pack-elasticsearch@91e4af140d
2017-05-01 11:30:19 -07:00
Jason Tedor bec3102e06 Upgrade Netty to 4.1.10.Final
This commit upgrades the Netty dependency from version 4.1.9.Final to
version 4.1.10.Final.

Relates elastic/x-pack-elasticsearch#1262

Original commit: elastic/x-pack-elasticsearch@aac7aa351c
2017-05-01 10:25:42 -04:00
Jason Tedor 5387472834 Adjust checkstyle suppressions to 140-column limit
We are back to having a 140-column limit. While at some point we will
apply auto-formatting tools to the code base, that is a down-the-road
thing and adjusting the suppressions now shaves minutes off the build
time.

Relates elastic/x-pack-elasticsearch#1260

Original commit: elastic/x-pack-elasticsearch@9a96aec9e6
2017-04-30 12:11:37 -04:00
Yannick Welsch a9aa3e2329 Revert "Test: get more information when sporadic NPE"
This reverts commit elastic/x-pack-elasticsearch@3e08725fc7.

Original commit: elastic/x-pack-elasticsearch@dc85c2b194
2017-04-29 10:51:35 +02:00
Yannick Welsch 308d5e0ae3 Don't call ClusterService.state() in a ClusterStateUpdateTask
The current state is readily available as a parameter

Original commit: elastic/x-pack-elasticsearch@a09929aa82
2017-04-29 10:50:56 +02:00
Yannick Welsch e3834da05e [TEST] Wait for trial license to be generated before running LicensesManagerServiceTests
Many of the tests assume that the trial license has already been generated before the test gets to run. As this is asynchronously triggered upon node
startup, however, there is no guarantee that trial license generation has completed before the tests get to execute, leading to null values when
checking clusterService.state().metaData().custom(LicensesMetaData.TYPE).

Original commit: elastic/x-pack-elasticsearch@d909c9ba95
2017-04-29 10:50:56 +02:00
Lisa Cawley 600aa8fc5d [DOCS] Add role=screenshot to format graphics (elastic/x-pack-elasticsearch#1259)
* [DOCS] Added role attribute to ML screenshots

* [DOCS] Fixing screenshot role in ML tutorial

Original commit: elastic/x-pack-elasticsearch@232a0632a0
2017-04-28 21:27:04 -07:00
Lisa Cawley d77f20b14e [DOCS] Update ML tutorial with latest sample data (elastic/x-pack-elasticsearch#1258)
* [DOCS] Updated mappings in ML sample data

* [DOCS] Update ML tutorial with latest sample data

Original commit: elastic/x-pack-elasticsearch@37fd75aed5
2017-04-28 12:31:24 -07:00
Lisa Cawley c4be95ae31 [DOCS] Remove unnecessary ML release notes file (elastic/x-pack-elasticsearch#1257)
Original commit: elastic/x-pack-elasticsearch@e3d2c66090
2017-04-28 11:48:08 -07:00
Lisa Cawley 4669a823cc [DOCS] Added ML sample data URLs (elastic/x-pack-elasticsearch#1256)
Original commit: elastic/x-pack-elasticsearch@528a32f26f
2017-04-28 10:28:37 -07:00
Dimitrios Athanasiou ee5e66bb0d [ML] Make updateDatafeed var in UpdateDatafeedAction volatile
Original commit: elastic/x-pack-elasticsearch@d8461d0960
2017-04-28 17:58:55 +01:00
Dimitris Athanasiou 5a70eac6e8 [ML] Delete and Update datafeed actions should use current cluster state (elastic/x-pack-elasticsearch#1254)
relates elastic/x-pack-elasticsearch#1253

Original commit: elastic/x-pack-elasticsearch@bdf695e694
2017-04-28 17:53:59 +01:00
Lisa Cawley 435338e6b9 [DOCS] Fix ML API example for elastic/x-pack-elasticsearch#1180 (elastic/x-pack-elasticsearch#1252)
Original commit: elastic/x-pack-elasticsearch@5290dd0a89
2017-04-28 09:14:53 -07:00
Nik Everett 459b77478e Test: get more information when sporadic NPE
LicenseManagerServiceTests sometimes fails in jenkins, but fairly
rarely. We don't get useful logs when it does. This cranks up
the log level and adds some more assertions so we can better track
down where the failure comes from.

Relates to elastic/x-pack-elasticsearch#222

Original commit: elastic/x-pack-elasticsearch@3e08725fc7
2017-04-28 11:34:48 -04:00
Lisa Cawley 68c3a94c35 [DOCS] Add ML limitations (elastic/x-pack-elasticsearch#1229)
* [DOCS] Add ML limitations

* [DOCS] Address feedback about ML limitations

* [DOCS] Change ML limitations capitalization

Original commit: elastic/x-pack-elasticsearch@41682d8d93
2017-04-28 08:13:47 -07:00
David Roberts 892d803a6a [ML] When putting a datafeed use runas user for index privilege check (elastic/x-pack-elasticsearch#1245)
When a user creates a datafeed, as well as checking they have permission
to create a datafeed we also check that they have permission to search the
indices they've configured the datafeed to search.

Previously this second check was erroneously done for the user who issued
the put_datafeed request, whereas it should be done as the runas user for
that request.

Original commit: elastic/x-pack-elasticsearch@4c35204c66
2017-04-28 13:38:53 +01:00
Clinton Gormley e61224778f Fixed bad link
Original commit: elastic/x-pack-elasticsearch@f7ab965eba
2017-04-28 12:35:07 +02:00
Dimitrios Athanasiou 57382390ab [TEST] Delete job from DatafeedJobsIT.testRealtime in clean up
relates elastic/x-pack-elasticsearch#1246

Original commit: elastic/x-pack-elasticsearch@9ca07f1308
2017-04-28 11:00:51 +01:00
Yannick Welsch c551bcba5c Separate publishing from applying cluster states
Companion commit to elastic/elasticsearch#24236

Original commit: elastic/x-pack-elasticsearch@d685478f5d
2017-04-28 09:35:20 +02:00
Lisa Cawley dd3578aaf0 [DOCS] Refresh ML screenshots (elastic/x-pack-elasticsearch#1242)
Original commit: elastic/x-pack-elasticsearch@e2341e49ea
2017-04-27 17:15:57 -07:00
Zachary Tong 6147e2ba6a Remove now-unnecessary cast
Original commit: elastic/x-pack-elasticsearch@fcbbe5a919
2017-04-27 17:16:30 -04:00
Dimitrios Athanasiou f341b336a5 [ML] Correctly pass job_id from job task to logger
Original commit: elastic/x-pack-elasticsearch@1c73bccaec
2017-04-27 20:02:15 +01:00
Lisa Cawley 485be502f4 [DOCS] Doc build fixes and edits for elastic/x-pack-elasticsearch#1237 (elastic/x-pack-elasticsearch#1241)
* [DOCS] Fixing doc build error

* [DOCS] Edits on ML content for elastic/x-pack-elasticsearch#1237

Original commit: elastic/x-pack-elasticsearch@cd4d404dee
2017-04-27 11:22:59 -07:00
Sophie Chang ffb3bb6493 [DOCS] Overall review (elastic/x-pack-elasticsearch#1237)
* [DOCS] Overall review

* [DOCS] General review

* [DOCS] typo

* [DOCS] Fix for processed_record_count with aggs

* [DOCS] Added latency tbd

Original commit: elastic/x-pack-elasticsearch@9e8cf664c1
2017-04-27 11:22:42 -07:00
Lisa Cawley 642b1f7c19 [DOCS] More edits for ML Getting Started (elastic/x-pack-elasticsearch#1238)
Original commit: elastic/x-pack-elasticsearch@69be11bfd2
2017-04-27 09:15:50 -07:00
Dimitrios Athanasiou b796388431 [ML] Delete intervening results after now as well
When we revert to snapshot, if we delete intervening results
we should delete with an open end on the time range for the
case when future data has been posted to the job.

Original commit: elastic/x-pack-elasticsearch@c3f5e8f19e
2017-04-27 16:38:23 +01:00
Nik Everett 677ea404f7 Remove most usages of 1-arg Script ctor (elastic/x-pack-elasticsearch#1207)
The one argument ctor for `Script` creates a script with the
default language but most usages of are for testing and either
don't care about the language or are for use with
`MockScriptEngine`. This replaces most usages of the one argument
ctor on `Script` with calls to `ESTestCase#mockScript` to make
it clear that the tests don't need the default scripting language.

Original commit: elastic/x-pack-elasticsearch@c1d05b7357
2017-04-27 11:35:42 -04:00
Sophie Chang aa7d94ec44 [DOCS] Review getting started (elastic/x-pack-elasticsearch#1219)
* [DOCS] Initial review of getting started

* [DOCS] Completed review of getting started

Original commit: elastic/x-pack-elasticsearch@a4b800b59b
2017-04-27 08:14:17 -07:00
Yannick Welsch 779e8f6771 [TEST] Reenable IndicesStatsTests
Commit elastic/x-pack-elasticsearch@4165beb90 should have fixed the test failures.

Original commit: elastic/x-pack-elasticsearch@67b8aac14c
2017-04-27 17:04:45 +02:00
Jay Modi 1d08b4d1fb Rest endpoints for token based access (elastic/x-pack-elasticsearch#1235)
This commit adds rest endpoints for the creation of a new token and invalidation of an existing
token. This builds upon the functionality that was introduced in elastic/x-pack-elasticsearch#1029.

relates elastic/x-pack-elasticsearch#8

Original commit: elastic/x-pack-elasticsearch@d56611dfa3
2017-04-27 11:04:31 -04:00
Jay Modi f7fb02f21f Ensure we always respect a user specified filter in the AD realm (elastic/x-pack-elasticsearch#1161)
When the active directory realm was refactored to add support for authenticating against multiple
domains, only the default authenticator respected the user_search.filter setting. This commit moves
this down to the base authenticator and also changes the UPN filter to not include sAMAccountName
in the filter.

Original commit: elastic/x-pack-elasticsearch@d2c19c9bee
2017-04-27 10:20:59 -04:00
Dimitrios Athanasiou 4f12f04c65 [TEST] Fix line width checkstyle error
Original commit: elastic/x-pack-elasticsearch@0d62752c0e
2017-04-27 14:03:07 +01:00
Dimitrios Athanasiou 4c9b4132c9 [TEST] Refactor ML integration test framework
- Removes need to handle exception from action methods
- Clearly renames DatafeedJobIT to DatafeedJobsRestIT to distinguish
  from DatafeedJobsIT
- Refactors DatafeedJobsIT to reuse MlNativeAutodetectIntegTestCase

Original commit: elastic/x-pack-elasticsearch@5bd0c01391
2017-04-27 13:43:28 +01:00
Tim Vernum ddf5fd68c2 Add ClusterSearchShardsAction to "read_cross_cluster" privilege (elastic/x-pack-elasticsearch#1231)
Cross cluster search uses ClusterSearchShardsAction under the covers.
Without this change, you would need both "read_cross_cluster" and "view_index_metadata" privilegs in order to have permission to execute searches from a remote cluster.

Original commit: elastic/x-pack-elasticsearch@65a6aff329
2017-04-27 22:39:13 +10:00
Dimitrios Athanasiou 7e62eb6255 [TEST] Wait for job to close from DatafeedJobsIT.testLookbackOnly
relates elastic/x-pack-elasticsearch#1228

Original commit: elastic/x-pack-elasticsearch@77055bd128
2017-04-27 12:57:50 +01:00