Commit Graph

4899 Commits

Author SHA1 Message Date
Areek Zillur 278496d5e5 x-pack changes for removing deprecated _suggest endpoint (elasticsearchelastic/elasticsearch#22203)
Original commit: elastic/x-pack-elasticsearch@b9bba20fae
2016-12-16 13:06:35 -05:00
David Kyle aacb6f3949 Move JobResultsPersister.getJobIndexName to new class AnomalyDetectorsIndex (elastic/elasticsearch#567)
Original commit: elastic/x-pack-elasticsearch@bf54dd56a9
2016-12-16 17:47:52 +00:00
Martijn van Groningen e3ec908828 run with native process by default
Original commit: elastic/x-pack-elasticsearch@c73beec522
2016-12-16 15:21:11 +01:00
Martijn van Groningen eaca8fb06a Stop local allocated jobs before closing node.
Original commit: elastic/x-pack-elasticsearch@ad3ed7c86a
2016-12-16 14:59:35 +01:00
David Roberts c43a9ba1dd Fix intermittent failure of renormalization (elastic/elasticsearch#558)
The synchronization was flawed

Original commit: elastic/x-pack-elasticsearch@a968c68c7d
2016-12-16 13:45:47 +00:00
Daniel Mitterdorfer d3c1fbe002 Enable strict duplicate checks for all XContent types
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default for all XContent types (not only JSON).

We have also changed the name of the system property to disable this feature
from `es.json.strict_duplicate_detection` to the now more appropriate name
`es.xcontent.strict_duplicate_detection`.

Relates elastic/elasticsearchelastic/elasticsearch#19614
Relates elastic/elasticsearchelastic/elasticsearch#22073

Original commit: elastic/x-pack-elasticsearch@c92e2a6c55
2016-12-16 13:57:55 +01:00
Colin Goodheart-Smithe 5e6c63bc51 Removes warnings
Original commit: elastic/x-pack-elasticsearch@59626a3a3a
2016-12-16 12:35:45 +00:00
Alexander Reelsen 0d9cb5f7bd Docs: Updating execution phases of watch execution
Closes elastic/elasticsearch#4079

Original commit: elastic/x-pack-elasticsearch@658ba0a7f7
2016-12-16 11:30:27 +01:00
Dimitris Athanasiou c932588c68 Repackage scheduler classes (elastic/elasticsearch#559)
* Repackage scheduler classes

* Rename and move scheduler status tests

Original commit: elastic/x-pack-elasticsearch@a9d7cc4e48
2016-12-16 09:50:06 +00:00
Daniel Mitterdorfer e3302da589 Make boolean conversion strict
This PR removes all leniency in the conversion of Strings to booleans: "true"
is converted to the boolean value `true`, "false" is converted to the boolean
value `false`. Everything else raises an error.

Original commit: elastic/x-pack-elasticsearch@6400f18911
2016-12-15 19:06:59 +01:00
Boaz Leskes b2972a142c Remove Cluster State Status (elastic/elasticsearch#4313)
A companion PR for https://github.com/elastic/elasticsearch/pull/21817

Original commit: elastic/x-pack-elasticsearch@392235877f
2016-12-15 17:07:02 +01:00
David Kyle 8d5c8b24d6 Make ModelDebugOutput fields snake case (elastic/elasticsearch#555)
* snake_case model debug output fields

* Rename CategorizerState and BucketInfluencer types to snake_case

Original commit: elastic/x-pack-elasticsearch@da94dc7ec1
2016-12-15 15:42:38 +00:00
David Kyle 8e14aee978 Audit message were being persisted with the type ‘audit_activity’ (elastic/elasticsearch#554)
Original commit: elastic/x-pack-elasticsearch@7c00495c98
2016-12-15 15:31:11 +00:00
Dimitris Athanasiou 8841a97659 Disallow non alphanumeric chars from start or end in IDs (elastic/elasticsearch#552)
Closes elastic/elasticsearch#550

Original commit: elastic/x-pack-elasticsearch@ca0ca9f4da
2016-12-15 13:49:07 +00:00
Dimitrios Athanasiou bc8d966690 Fix integration tests
Original commit: elastic/x-pack-elasticsearch@a5955cfde3
2016-12-15 12:03:51 +00:00
David Kyle b97610f245 Streamed boolean values can on be 0 or 1
Original commit: elastic/x-pack-elasticsearch@16ffa0727b
2016-12-15 12:01:09 +00:00
Alexander Reelsen e6ee905931 Watcher: Remove unused class
Original commit: elastic/x-pack-elasticsearch@ecd48b7914
2016-12-15 11:33:36 +01:00
Ryan Ernst 6d4d599f91 Build: Convert xplugins to use new extra projects setup (elastic/elasticsearch#4175)
* Build: Convert xplugins to use new extra projects setup

This change makes the gradle initialization for xplugins look in the
correct location for elasticsearch, which is now as a sibling of an
elasticsearch-extra directory, with x-plugins as a child of the extra
directory.

The elasticsearch side of this change is
elastic/elasticsearchelastic/elasticsearch#21773. This change will enable renaming x-plugins
to x-pack, see elastic/elasticsearch#3643.

Original commit: elastic/x-pack-elasticsearch@09398aea5a
2016-12-14 15:02:28 -08:00
Martijn van Groningen faf54d4952 renamed test classes
Original commit: elastic/x-pack-elasticsearch@5296de69d7
2016-12-14 19:52:10 +01:00
Martijn van Groningen 985bdf6cb1 Changed rest start scheduler api to wait with returning a response until the the scheduler status has been set to STARTED and
changed stop scheduler api to wait with returning a response until the scheduler status has been set to STOPPED.

Original commit: elastic/x-pack-elasticsearch@e20fcd1ae9
2016-12-14 19:50:02 +01:00
Martijn van Groningen 3b487a268b Add validation for stop scheduler api
Original commit: elastic/x-pack-elasticsearch@4571bbacd9
2016-12-14 19:28:12 +01:00
David Kyle f085b935f6 Custom all mapping for interesting result fields (elastic/elasticsearch#535)
* Check use of mappings

* Add unit tests for JobProvider.createJobRelatedIndices

* Remove ‘index: no’ from mappings as no longer required

The entire type mapping has ‘enabled: false’

* Restore “index.analysis.analyzer.default.type” setting

* Remove include_in_all from nested mappings

* Add audit and usage mappings to the job index

* Revert ‘Restore “index.analysis.analyzer.default.type” setting’

Original commit: elastic/x-pack-elasticsearch@c7d62e0c7e
2016-12-14 16:32:36 +00:00
Nik Everett 597ca9275e Finish centralizing XContentParser creation in tests
Now we're ready to add `NamedXContentRegistry`.

Original commit: elastic/x-pack-elasticsearch@c4f6c7c3d6
2016-12-14 11:19:09 -05:00
David Roberts 0a45d846df Add back native process normalizer functionality (elastic/elasticsearch#540)
This builds on PR elastic/elasticsearch#526 to get normalization working end-to-end using the
native normalizer process.

The ShortCircuitingRenormalizer class is basically doing what the old
BlockingQueueRenormaliser class did but using the ES threadpool instead
of its own thread.

Also fixed a bug where the C++ was calling the score field of partition_score
documents normalized_probability but the Java was calling it anomaly_score.

Original commit: elastic/x-pack-elasticsearch@d4cecae150
2016-12-14 16:06:01 +00:00
Dimitris Athanasiou 4732223214 Split SchedulerConfig from Job (elastic/elasticsearch#541)
Original commit: elastic/x-pack-elasticsearch@32a55da726
2016-12-14 14:10:44 +00:00
David Kyle 7efdbd9320 Fix the reported number of bytes read (elastic/elasticsearch#544)
Original commit: elastic/x-pack-elasticsearch@a98902f309
2016-12-14 13:39:00 +00:00
Daniel Mitterdorfer 911aec0502 Enable strict duplicate checks for JSON content
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default. This ensures that JSON keys are always unique. While this has
a performance impact, benchmarking has indicated that the typical drop in
indexing throughput is around 1 - 2%.

As a last resort, we allow users to still disable strict duplicate checks
by setting `-Des.json.strict_duplicate_detection=false` which is
intentionally undocumented.

Relates elastic/elasticsearchelastic/elasticsearch#19614
Relates elastic/elasticsearchelastic/elasticsearch#22073

Original commit: elastic/x-pack-elasticsearch@47055336e3
2016-12-14 10:29:21 +01:00
Daniel Mitterdorfer feabaaf1d6 Add missing import
Original commit: elastic/x-pack-elasticsearch@fbc8eb7ac6
2016-12-14 09:34:55 +01:00
Daniel Mitterdorfer 83240e25b4 Merge remote-tracking branch 'origin/master' into json_strict_duplicate_checks
Original commit: elastic/x-pack-elasticsearch@af25e460d0
2016-12-14 08:53:45 +01:00
Nik Everett d4939e1972 Start centralizing XContentParser creation in tests
Original commit: elastic/x-pack-elasticsearch@65be7ecd27
2016-12-13 20:28:19 -05:00
Spencer c3d8fd4191 Merge pull request elastic/elasticsearch#3259 from spalger/upgrade/eslint
[ui] Upgrade eslint

Original commit: elastic/x-pack-elasticsearch@2e47b105ff
2016-12-13 17:39:25 -07:00
Jason Tedor 3f5d24ae55 Add shutdown hook for closing CLI commands
This commit adapts some CLI command tests for some changes in core.

Relates elastic/elasticsearch#4342

Original commit: elastic/x-pack-elasticsearch@1e48c1404f
2016-12-13 19:10:35 -05:00
Alexander Reelsen 1e7985563a Tests: Make MockWebServer thread-safe (elastic/elasticsearch#4351)
The new MockWebServer assumed that responses are inserted at the beginning
and removed later on. This was not thread safe.

Also this fixes a bug in the HttpExporterIT where there was no wait time
for a bulk request, even though the request execution is asynchronous.

Closes elastic/elasticsearch#4335

Original commit: elastic/x-pack-elasticsearch@11f31f68bd
2016-12-13 19:34:33 +01:00
Zachary Tong b386ed33a1 Add ability to get all lists, ensure params are mutually exclusive (elastic/elasticsearch#514)
Also removes some unnecessary stuff in validate()

Related elastic/elasticsearch#291

Original commit: elastic/x-pack-elasticsearch@d9af2d3a30
2016-12-13 11:38:11 -05:00
Zachary Tong 6ef954e3f0 Add mutually-exclusive param validation to GetCategories (elastic/elasticsearch#512)
Also adds body parsing to the REST action, which seemed to be missing (but supported because of the POST)

Closes elastic/elasticsearch#430

Original commit: elastic/x-pack-elasticsearch@e7455ffbb6
2016-12-13 11:37:46 -05:00
Daniel Mitterdorfer 8be3250601 Merge remote-tracking branch 'origin/master' into json_strict_duplicate_checks
Original commit: elastic/x-pack-elasticsearch@a010b1b4e9
2016-12-13 16:22:47 +01:00
Daniel Mitterdorfer 7cb0771c46 Use assume* instead of custom skipping logic
Original commit: elastic/x-pack-elasticsearch@0f3a01735f
2016-12-13 16:21:46 +01:00
Martijn van Groningen 3581fc9d91 removed redundant private method
Original commit: elastic/x-pack-elasticsearch@dbc377d6b4
2016-12-13 13:35:14 +01:00
Martijn van Groningen d15933ac32 test: move java rest test to yaml rest test.
These test are just checking response / status code. It is cleaner to have these tests as yaml tests.

Original commit: elastic/x-pack-elasticsearch@61c323059e
2016-12-13 12:13:24 +01:00
Luca Cavanna 1c846dd893 Stop execution once destructive operations check has failed (elastic/elasticsearch#4337)
Otherwise we do return an error, but we also go ahead and open/close/delete the indices anyways.

Original commit: elastic/x-pack-elasticsearch@176eca4cff
2016-12-13 11:22:48 +01:00
David Kyle 9f73f047eb Test for job existence before updating its state (elastic/elasticsearch#532)
* Test for job existence before updating its state

* Add unit tests covering expected missing job exceptions

Original commit: elastic/x-pack-elasticsearch@bcd270dafd
2016-12-13 09:24:16 +00:00
Alexander Reelsen b57c4f6ebe Watcher: Remove in memory watch store (elastic/elasticsearch#4201)
In order to prepare to the distributed watch execution, this commit
removes the in memory watch store.

Whenever a watch is needed now, a get request is executed and the parsing
is done. This happens when

* Put
* Get
* Ack
* Activate/Deactivate
* Execute

Note: This also means there are no usage stats currently regarding
the watch count, because we would need to execute a query. This would
require the usage stats to be async, see elastic/elasticsearch#3569

Another advantage is, that there is no dirty flag in the watch itself
needed anymore, because the watch is always the latest. Also write
operations store immediately and dont leave anything in memory.

Also ActionListener.wrap() was used a lot instead of more verbose anonmyous
inner classes.

Original commit: elastic/x-pack-elasticsearch@c47465b47c
2016-12-13 08:54:03 +01:00
Tim Vernum 7192c46307 Define explicit settings for security realms/ssl (elastic/elasticsearch#4311)
Modified the definition and loading of settings in Security to provide early detection and failure of invalid (unrecognised or syntactically invalid) settings.
Also consolidates the number of places where settings were defined.

Each realm now defines its supported settings.
This is facilitated for custom realms via a new "getRealmSettings" method on XPackExtension.
The realm group setting performs validation of the child settings with reference to the "type".
For backwards compatibility reasons, realm types that have no configuration defined, will be accepted during validation, but may fail at realm creation time.

All SSL settings have been centralised into SSLConfigurationSettings, which supports a variable "prefix" to accommodate the multiple places we define SSL config.

HTTP Proxy settings are explicitly defined rather than being a generic group.

Where possible all security settings now reference a Setting object, and there are less magic strings scattered in the code.

Closes: elastic/elasticsearch#3965

Original commit: elastic/x-pack-elasticsearch@2c76a137a9
2016-12-13 16:14:02 +11:00
David Kyle bca06f0ad7 Fix forbidden API errors and logger usage check (elastic/elasticsearch#533)
Original commit: elastic/x-pack-elasticsearch@e7bdaa8b0a
2016-12-12 17:31:39 +00:00
David Kyle 1d67c6ea1d Fix build: rename import Renormali[s]er -> Renormali[z]er
Original commit: elastic/x-pack-elasticsearch@70fa67aaf7
2016-12-12 17:08:39 +00:00
David Kyle 2302dc78ba AutodetectResultProcessor Integration Test (elastic/elasticsearch#516)
* Add results processor integration test

* Integration tests for AutodetectResultProcessor

Original commit: elastic/x-pack-elasticsearch@19e7ec48dd
2016-12-12 16:55:20 +00:00
David Kyle dae0b5625d Add yaml tests assert a 404 is returned when the job id isn’t recognised (elastic/elasticsearch#531)
For open, close, _flush and posting data

Original commit: elastic/x-pack-elasticsearch@be2c29c409
2016-12-12 16:00:20 +00:00
David Kyle 300d7c67d2 Fix failing integration tests
Incorrect job id was used

Original commit: elastic/x-pack-elasticsearch@625a1616ee
2016-12-12 13:03:09 +00:00
David Kyle 3829a83864 Check job exists and return a 404 on flush (elastic/elasticsearch#529)
Original commit: elastic/x-pack-elasticsearch@7721d4510a
2016-12-12 12:50:40 +00:00
David Kyle 46938b281c Check job exists before opening & closing (elastic/elasticsearch#530)
Original commit: elastic/x-pack-elasticsearch@037ea30a12
2016-12-12 12:50:29 +00:00