Commit Graph

4416 Commits

Author SHA1 Message Date
jaymode 9288401767 test: do not use security manager for client tests
Original commit: elastic/x-pack-elasticsearch@1200a1b757
2016-12-21 14:46:18 -05:00
Daniel Mitterdorfer b973d10685 Improve BWC
Original commit: elastic/x-pack-elasticsearch@f84cb5b1fa
2016-12-21 16:43:55 +01:00
Nik Everett 483711ed20 Handle removal of IndiciesQueriesRegistry
It has been replaced by `XContentParser#namedObject`.

Original commit: elastic/x-pack-elasticsearch@fd316791a2
2016-12-21 08:01:49 -05:00
Daniel Mitterdorfer fa1219fd13 Merge remote-tracking branch 'origin/master' into strict-booleans
Original commit: elastic/x-pack-elasticsearch@6fbbaf19dd
2016-12-21 08:47:45 +01:00
Jason Tedor 1178112c4a Skip pre-built transport client test on JDK 9
This test will currently not pass because Netty tries to do unsafe
operations that JDK 9 shoots down.

Relates elastic/elasticsearch#4430

Original commit: elastic/x-pack-elasticsearch@42aa250d4c
2016-12-20 20:25:41 -05:00
Tim B f45c24faf2 Fix x-pack readme to reflect project name and dev-setup changes (elastic/elasticsearch#4402)
* Update readme to reflect new dev setup directory structure.

* Fix typo in elasticsearch-extra path in readme

* Update gradle exception for x-pack directory structure.

* Make directory path where x-pack must be checked out explicit in the gradle exception

Original commit: elastic/x-pack-elasticsearch@91f1d04542
2016-12-20 12:43:32 -06:00
Nik Everett b2dc1bdca3 Introduce NamedXContentRegistry (elastic/elasticsearch#4399)
This "super registry" will eventually replace things like
`IndiciesQueriesRegistry` but for now it is just another thing
to plumb across requests.

Original commit: elastic/x-pack-elasticsearch@da26a42b36
2016-12-20 11:05:42 -05:00
Alexander Reelsen 6f7a065605 Watcher: Parse index action results to prevent exceptions (elastic/elasticsearch#4424)
When the index action is used to do some bulk indexing, the single
items of the response were not checked to have been indexed successful.
This could lead to NPEs due to an index response being null when the index
operation had failed. The action was still logged as a success though.

This commit only returns SUCCESS for the action, if all items were indexed
successfully. If all items failed, the result will be FAILED as well. Lastly
the result status PARTIAL_FAILURE is used if there were successful and unsuccessful
index operations.

Additionally some minor cleanups happened, like changing equals/hashcode.

Closes elastic/elasticsearch#4416

Original commit: elastic/x-pack-elasticsearch@692687e1af
2016-12-20 16:53:12 +01:00
Dimitrios Liappis 17c3164979 [ci] Fix bug with -Dtests.jvm.argline
The special `-Dtests.jvm.argline` params needed for jdk-9 builds do not
get passed correctly if enclosed within single quotes.

Fix jdk9 target for `dev-tools/ci` script to correctly pass the
-Dtests.jvm.argline parameters.

Relates:  elastic/elasticsearch#4428 

Original commit: elastic/x-pack-elasticsearch@6cd329b8da
2016-12-20 16:35:18 +02:00
Alexander Reelsen 0f8e6354f6 Watcher: Allow deletion of watches regardless of license (elastic/elasticsearch#4414)
The user might want to delete watches on expiry to prevent
further execution, without any actions being executed.

Closes elastic/elasticsearch#4413

Original commit: elastic/x-pack-elasticsearch@244a1b9cd8
2016-12-20 15:26:06 +01:00
Yannick Welsch 68bfc90cea Let ClusterStateObserver only hold onto state that's needed for change detection
Companion commit for elastic/elasticsearchelastic/elasticsearch#21631

Original commit: elastic/x-pack-elasticsearch@3333480707
2016-12-20 15:23:00 +01:00
Alexander Reelsen b994f16f4a Watcher: Ensure non duplicate Wid ids (elastic/elasticsearch#4423)
Removing the WatchLockService could result in duplication of wids, because of a wrong
call to replace underscores with dashes. As UUIDs.createBase64UUID() can contain underscores
but they are kind of reserved in the Wid class due to handling of watch ids, this just uses
the toString() representation of a random UUID.

Closes elastic/elasticsearch#4422

Original commit: elastic/x-pack-elasticsearch@dceb01ae5e
2016-12-20 14:02:00 +01:00
Daniel Mitterdorfer b9a38d9b97 Enable trace logging for LocalExporterTests#testSimpleExport()
This test fails due to a changing cluster state(?).  The
test checks that a local exporter is ready and then continues.
However, during the test, we see output similar to:

skipping exporter [_local] as it isn't ready yet

Which indicates that the cluster state has changed and the
exporter does not return a bulk anymore. Hence, the test is
failing although at one point in time it returned a bulk.

By enabling trace logging we should be able to find out
what's going on.

Original commit: elastic/x-pack-elasticsearch@d7e2200dd9
2016-12-20 08:54:26 +01:00
Ryan Ernst 789df7d2fa Internal: Update signatures for EnvironmentAwareCommand (elastic/elasticsearch#4367)
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#22175

Original commit: elastic/x-pack-elasticsearch@4359cb1947
2016-12-19 15:23:53 -08:00
javanna adf22c3d24 Adapt to strict parsing mode removal
Original commit: elastic/x-pack-elasticsearch@2d6276a60f
2016-12-19 19:39:08 +01:00
Dimitrios Liappis 8a3181ceed [ci] Add jdk9 target to the dev-tools/ci script
To add support for jdk9 tests for x-pack we need a number of parameters
for the jvm, passed in the gradle command. To resolve issues with arg
passing through bash variables and improve readability, switch to an
array construct for storing the arguments.

Add new jdk9 target to allow execution of jdk-9-ea based tests in the CI
infra.

Relates: elastic/elasticsearch#4415 
Relates: https://github.com/elastic/infra/issues/1573#issuecomment-267832307

Original commit: elastic/x-pack-elasticsearch@b41e5f4b98
2016-12-19 19:53:00 +02:00
Alexander Reelsen 79a8f27569 Watcher: Remove WatchLockService (elastic/elasticsearch#4365)
The watch lock service is not really needed, as there is already
a data structure that has information about the currently executing
watches, that can be consulted before executed.

This change will now check, if there is already a watch running with
the current id. If there is not, execution will happen as usual. If
there is however, than a watch record will be created, stating that
the watch is currently being executed - which means that it is either
being executed or in the list of planned executions.

This way users can check in the watch history, if a watch has been executed
more often than it should.

In order to easily search for this, a new execution state called
`NOT_EXECUTED_ALREADY_QUEUED` has been added.

Original commit: elastic/x-pack-elasticsearch@867acec3c3
2016-12-19 17:33:48 +01:00
Daniel Mitterdorfer 2cb41e690d Address initial review comments
Original commit: elastic/x-pack-elasticsearch@bcd6433afb
2016-12-19 13:27:40 +01:00
Daniel Mitterdorfer 24a4290379 Merge remote-tracking branch 'origin/master' into strict-booleans
Original commit: elastic/x-pack-elasticsearch@32a533f38f
2016-12-19 12:58:30 +01:00
Daniel Mitterdorfer 7a652fa090 Merge pull request elastic/elasticsearch#4398 from danielmitterdorfer/xcontent_strict_duplicate_checks
Enable strict duplicate checks for all XContent types

Original commit: elastic/x-pack-elasticsearch@ad249c2e65
2016-12-19 09:30:19 +01:00
Tim Vernum df7c528c66 Change classification of event_type in index audit trail (elastic/elasticsearch#4375)
In LoggingAuditTrail and the audit-event filtering, we distinguish between single-realm authentication failures ("realm_authentication_failure") and global failures ("authentication_failure").

Update the message output of IndexAuditTrail to reflect this distinction.

Original commit: elastic/x-pack-elasticsearch@56802ae2df
2016-12-19 14:39:21 +11:00
Simon Willnauer c9cfedc3cb [TEST] check if .watches index / alias exists before creating it
Original commit: elastic/x-pack-elasticsearch@74d85121eb
2016-12-18 10:17:43 +01:00
Areek Zillur 278496d5e5 x-pack changes for removing deprecated _suggest endpoint (elasticsearchelastic/elasticsearch#22203)
Original commit: elastic/x-pack-elasticsearch@b9bba20fae
2016-12-16 13:06:35 -05:00
Daniel Mitterdorfer d3c1fbe002 Enable strict duplicate checks for all XContent types
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default for all XContent types (not only JSON).

We have also changed the name of the system property to disable this feature
from `es.json.strict_duplicate_detection` to the now more appropriate name
`es.xcontent.strict_duplicate_detection`.

Relates elastic/elasticsearchelastic/elasticsearch#19614
Relates elastic/elasticsearchelastic/elasticsearch#22073

Original commit: elastic/x-pack-elasticsearch@c92e2a6c55
2016-12-16 13:57:55 +01:00
Alexander Reelsen 0d9cb5f7bd Docs: Updating execution phases of watch execution
Closes elastic/elasticsearch#4079

Original commit: elastic/x-pack-elasticsearch@658ba0a7f7
2016-12-16 11:30:27 +01:00
Daniel Mitterdorfer e3302da589 Make boolean conversion strict
This PR removes all leniency in the conversion of Strings to booleans: "true"
is converted to the boolean value `true`, "false" is converted to the boolean
value `false`. Everything else raises an error.

Original commit: elastic/x-pack-elasticsearch@6400f18911
2016-12-15 19:06:59 +01:00
Boaz Leskes b2972a142c Remove Cluster State Status (elastic/elasticsearch#4313)
A companion PR for https://github.com/elastic/elasticsearch/pull/21817

Original commit: elastic/x-pack-elasticsearch@392235877f
2016-12-15 17:07:02 +01:00
Alexander Reelsen e6ee905931 Watcher: Remove unused class
Original commit: elastic/x-pack-elasticsearch@ecd48b7914
2016-12-15 11:33:36 +01:00
Ryan Ernst 6d4d599f91 Build: Convert xplugins to use new extra projects setup (elastic/elasticsearch#4175)
* Build: Convert xplugins to use new extra projects setup

This change makes the gradle initialization for xplugins look in the
correct location for elasticsearch, which is now as a sibling of an
elasticsearch-extra directory, with x-plugins as a child of the extra
directory.

The elasticsearch side of this change is
elastic/elasticsearchelastic/elasticsearch#21773. This change will enable renaming x-plugins
to x-pack, see elastic/elasticsearch#3643.

Original commit: elastic/x-pack-elasticsearch@09398aea5a
2016-12-14 15:02:28 -08:00
Nik Everett 597ca9275e Finish centralizing XContentParser creation in tests
Now we're ready to add `NamedXContentRegistry`.

Original commit: elastic/x-pack-elasticsearch@c4f6c7c3d6
2016-12-14 11:19:09 -05:00
Daniel Mitterdorfer 911aec0502 Enable strict duplicate checks for JSON content
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default. This ensures that JSON keys are always unique. While this has
a performance impact, benchmarking has indicated that the typical drop in
indexing throughput is around 1 - 2%.

As a last resort, we allow users to still disable strict duplicate checks
by setting `-Des.json.strict_duplicate_detection=false` which is
intentionally undocumented.

Relates elastic/elasticsearchelastic/elasticsearch#19614
Relates elastic/elasticsearchelastic/elasticsearch#22073

Original commit: elastic/x-pack-elasticsearch@47055336e3
2016-12-14 10:29:21 +01:00
Daniel Mitterdorfer feabaaf1d6 Add missing import
Original commit: elastic/x-pack-elasticsearch@fbc8eb7ac6
2016-12-14 09:34:55 +01:00
Daniel Mitterdorfer 83240e25b4 Merge remote-tracking branch 'origin/master' into json_strict_duplicate_checks
Original commit: elastic/x-pack-elasticsearch@af25e460d0
2016-12-14 08:53:45 +01:00
Nik Everett d4939e1972 Start centralizing XContentParser creation in tests
Original commit: elastic/x-pack-elasticsearch@65be7ecd27
2016-12-13 20:28:19 -05:00
Spencer c3d8fd4191 Merge pull request elastic/elasticsearch#3259 from spalger/upgrade/eslint
[ui] Upgrade eslint

Original commit: elastic/x-pack-elasticsearch@2e47b105ff
2016-12-13 17:39:25 -07:00
Jason Tedor 3f5d24ae55 Add shutdown hook for closing CLI commands
This commit adapts some CLI command tests for some changes in core.

Relates elastic/elasticsearch#4342

Original commit: elastic/x-pack-elasticsearch@1e48c1404f
2016-12-13 19:10:35 -05:00
Alexander Reelsen 1e7985563a Tests: Make MockWebServer thread-safe (elastic/elasticsearch#4351)
The new MockWebServer assumed that responses are inserted at the beginning
and removed later on. This was not thread safe.

Also this fixes a bug in the HttpExporterIT where there was no wait time
for a bulk request, even though the request execution is asynchronous.

Closes elastic/elasticsearch#4335

Original commit: elastic/x-pack-elasticsearch@11f31f68bd
2016-12-13 19:34:33 +01:00
Daniel Mitterdorfer 8be3250601 Merge remote-tracking branch 'origin/master' into json_strict_duplicate_checks
Original commit: elastic/x-pack-elasticsearch@a010b1b4e9
2016-12-13 16:22:47 +01:00
Daniel Mitterdorfer 7cb0771c46 Use assume* instead of custom skipping logic
Original commit: elastic/x-pack-elasticsearch@0f3a01735f
2016-12-13 16:21:46 +01:00
Luca Cavanna 1c846dd893 Stop execution once destructive operations check has failed (elastic/elasticsearch#4337)
Otherwise we do return an error, but we also go ahead and open/close/delete the indices anyways.

Original commit: elastic/x-pack-elasticsearch@176eca4cff
2016-12-13 11:22:48 +01:00
Alexander Reelsen b57c4f6ebe Watcher: Remove in memory watch store (elastic/elasticsearch#4201)
In order to prepare to the distributed watch execution, this commit
removes the in memory watch store.

Whenever a watch is needed now, a get request is executed and the parsing
is done. This happens when

* Put
* Get
* Ack
* Activate/Deactivate
* Execute

Note: This also means there are no usage stats currently regarding
the watch count, because we would need to execute a query. This would
require the usage stats to be async, see elastic/elasticsearch#3569

Another advantage is, that there is no dirty flag in the watch itself
needed anymore, because the watch is always the latest. Also write
operations store immediately and dont leave anything in memory.

Also ActionListener.wrap() was used a lot instead of more verbose anonmyous
inner classes.

Original commit: elastic/x-pack-elasticsearch@c47465b47c
2016-12-13 08:54:03 +01:00
Tim Vernum 7192c46307 Define explicit settings for security realms/ssl (elastic/elasticsearch#4311)
Modified the definition and loading of settings in Security to provide early detection and failure of invalid (unrecognised or syntactically invalid) settings.
Also consolidates the number of places where settings were defined.

Each realm now defines its supported settings.
This is facilitated for custom realms via a new "getRealmSettings" method on XPackExtension.
The realm group setting performs validation of the child settings with reference to the "type".
For backwards compatibility reasons, realm types that have no configuration defined, will be accepted during validation, but may fail at realm creation time.

All SSL settings have been centralised into SSLConfigurationSettings, which supports a variable "prefix" to accommodate the multiple places we define SSL config.

HTTP Proxy settings are explicitly defined rather than being a generic group.

Where possible all security settings now reference a Setting object, and there are less magic strings scattered in the code.

Closes: elastic/elasticsearch#3965

Original commit: elastic/x-pack-elasticsearch@2c76a137a9
2016-12-13 16:14:02 +11:00
javanna 85679fcf19 Adapt to removal of support for empty queries
Original commit: elastic/x-pack-elasticsearch@87915a9cde
2016-12-12 13:34:30 +01:00
Daniel Mitterdorfer 050a36bb28 Address review comments in ES core
Original commit: elastic/x-pack-elasticsearch@925c78d4fe
2016-12-12 13:30:39 +01:00
Daniel Mitterdorfer b7d2ef0160 Merge remote-tracking branch 'origin/master' into json_strict_duplicate_checks
Original commit: elastic/x-pack-elasticsearch@601a663982
2016-12-12 12:36:38 +01:00
Alexander Reelsen 779eb44b66 Watcher: Cleanup - Replace list of integers with bitset (elastic/elasticsearch#4324)
In the triggered watch store a list of integers was returned to keep track
which watches had been successfully stored and thus should be executed.

This means, that an arraylist, plus autoboxing/unboxing needs to be done for
all the triggered watches. This data structure can easily be replaced with a BitSet,
resulting in much less objects being created or parsed - also it's a bit faster.

Original commit: elastic/x-pack-elasticsearch@e9fba67e34
2016-12-12 09:48:58 +01:00
Simon Willnauer 5716a20fd3 Fix compile issues after elastic/elasticsearchelastic/elasticsearch#22037
Original commit: elastic/x-pack-elasticsearch@c286839ba5
2016-12-10 11:05:58 +01:00
Nik Everett f6cb60204a Handle core moving some methods around
We're trying to move all the XContentParser creation into
`RestRequest`.

Original commit: elastic/x-pack-elasticsearch@87c05c4a9a
2016-12-09 20:01:53 -05:00
Stacey Gammon e321337d4c Merge pull request elastic/elasticsearch#4296 from Stacey-Gammon/save-rename-graph-update
Incorporate new save/rename flow for graph

Original commit: elastic/x-pack-elasticsearch@27d2031381
2016-12-09 08:49:10 -05:00
Jay Modi ac34c3c37f Remove deprecated methods from the Realm class
This change removes the deprecated methods from the realm class. These methods include
blocking authentication and lookup and the lookup supported method.

Original commit: elastic/x-pack-elasticsearch@cff21e21ee
2016-12-09 07:28:11 -05:00