Commit Graph

4056 Commits

Author SHA1 Message Date
Nik Everett 2aa0781971 Explicitly shut down watcher after tests
In our tests you have to explicitly shut down watcher rather than shut
down the node it is running on because of thread leak detection. Just
shutting down the node that it is running on will cause it to start up
on another node if there is another one running and then not properly
shut down. This is probably something that should be fixed in watcher
somehow but for now lets just be more careful with the tests.

Closes elastic/elasticsearch#2365
Closes elastic/elasticsearch#2588

Original commit: elastic/x-pack-elasticsearch@fb8a172972
2016-09-09 14:43:46 -04:00
javanna 5829d112d1 Move search ext section parsing to the coordinating node
Original commit: elastic/x-pack-elasticsearch@1ce38fcff9
2016-09-09 18:58:10 +02:00
Jason Tedor 592ca5ab7b Adjust to logger API changes in Elasticsearch
Previously core Elasticsearch had methods in a test class for removing
and adding appenders. However, these methods were moved to production
code. This commit adjusts x-plugins for this change.

Original commit: elastic/x-pack-elasticsearch@83e37ef65a
2016-09-09 09:15:53 -04:00
Martijn van Groningen b74f1e6cb2 watcher: Add limited capability to upgrade the source of a watcher upon startup.
This particular change focuses on upgrading the source of a watch when it comes to scripts that have no language specified explicitly.
The default language in version 5 changed to painless from whatever is specified in `script.default_lang` setting (this defaulted to groovy). In order to make sure that scripts in watcher remain to work we should rewrite the search source upon startup and set the legacy default language explicitly. The legacy script language is now controlled by `script.legacy.default_lang` setting and that defaults to groovy.

Changing the source upon startup should do the trick and only change the source of watches with scripts that don't have an explicit language set. For new watches the default language used in scripts is painless and because we now always serialize the language explicitly in scripts these watches won't be changed on startup.

 The upgrade logic added here tries to upgrade scripts in the following places in a watch:
 * script condition
 * script transform
 * any script defined inside of a search input

Original commit: elastic/x-pack-elasticsearch@4d578819eb
2016-09-09 14:26:24 +02:00
Britta Weber 4bf685cd31 [TEST] wait for pending tasks finished before testing cluster state collector
Original commit: elastic/x-pack-elasticsearch@d42c8c4bbe
2016-09-09 12:07:52 +02:00
Tanguy Leroux 6ca086b997 Fix line length in LoggingAuditTrailTests.java
Original commit: elastic/x-pack-elasticsearch@3f4185dce7
2016-09-09 10:50:43 +02:00
Colin Goodheart-Smithe 8b6d988cfb Security: Audit all HTTP requests
Adds a new audit event (authentication_success) which logs each request made to
the REST API along with the body of the request

Closes elastic/elasticsearch#912

Original commit: elastic/x-pack-elasticsearch@650b9d70c0
2016-09-09 09:28:27 +01:00
Joe Fleming c7aabce3df Merge pull request elastic/elasticsearch#2915 from w33ble/reporting-enhancement/sync-workflow
Synchronous workflow, functionality with Watcher

Original commit: elastic/x-pack-elasticsearch@8d4f9fb643
2016-09-08 16:30:53 -07:00
Joe Fleming 21532389dd Merge branch 'master' into reporting-enhancement/sync-workflow
Original commit: elastic/x-pack-elasticsearch@4228a6e229
2016-09-08 16:25:26 -07:00
Joe Fleming ecbe934e18 Merge pull request elastic/elasticsearch#3362 from w33ble/reporting-fix/user-permissions
Use user permissions in Reporting

Original commit: elastic/x-pack-elasticsearch@6717c6624c
2016-09-08 15:56:46 -07:00
Simon Willnauer 923949ec67 Downgrade hard bwc break to a TODO for now
Original commit: elastic/x-pack-elasticsearch@73c2a6ee3e
2016-09-08 22:05:36 +02:00
Simon Willnauer 5c407cc526 Revert "Remove bwc layer for TimeUnit parsing"
This reverts commit elastic/x-pack@c6b2bd2c12.

Original commit: elastic/x-pack-elasticsearch@b4fbfb8afe
2016-09-08 22:03:46 +02:00
Court Ewing 5defa5115d Merge pull request elastic/elasticsearch#3398 from epixa/3299-defaultreportingkey
reporting: Generate reporting.encryptionKey automatically

Original commit: elastic/x-pack-elasticsearch@2c754f0db9
2016-09-08 15:31:14 -04:00
Matt Bargar a547804b7f Merge pull request elastic/elasticsearch#3378 from Bargs/fixSecuritySection
Fix double registration of security management sections

Original commit: elastic/x-pack-elasticsearch@51adc9ce1c
2016-09-08 15:08:48 -04:00
Jason Tedor 9baf88dfaa Handle node validation exception when starting
Node#start can now throw a checked NodeValidationException; this commit
adapts x-plugins for this new API.

Relates elastic/elasticsearch#3381

Original commit: elastic/x-pack-elasticsearch@660c5ab8c1
2016-09-08 10:56:54 -04:00
Simon Willnauer 20fce3280e Remove bwc layer for TimeUnit parsing
Original commit: elastic/x-pack-elasticsearch@79145f279b
2016-09-08 16:43:46 +02:00
Court Ewing 3931ca0fc1 Merge pull request elastic/elasticsearch#3379 from epixa/3101-reportingunsavedstate
reporting: Require user to save changes for reporting

Original commit: elastic/x-pack-elasticsearch@cefa3a0be8
2016-09-08 10:28:00 -04:00
Tanguy Leroux 97182fefb9 Clean up XContentBuilder in X-Pack
This commit cleans most of the methods of XContentBuilder so that:
    - Jackson's convenience methods are used instead of our custom ones (ie field(String,long) now uses Jackson's writeNumberField(String, long) instead of calling writeField(String) then writeNumber(long))
    - null checks are added for all field names and values
    - methods are grouped by type in the class source
    - methods have the same parameters names
    - duplicated methods like field(String, String...) and array(String, String...) are removed
    - varargs methods now have the "array" name to reflect that it builds arrays
    - unused methods like field(String,BigDecimal) are removed
    - all methods now follow the execution path: field(String,?) -> field(String) then value(?), and value(?) -> writeSomething() method. Methods to build arrays also follow the same execution path.

Original commit: elastic/x-pack-elasticsearch@d83f3aa6e2
2016-09-08 14:17:15 +02:00
Joe Fleming 8b6bd14b18 Merge pull request elastic/elasticsearch#3284 from ycombinator/reporting/5.0/encrypt-headers
Encrypt headers in reporting job doc

Original commit: elastic/x-pack-elasticsearch@9f21f20add
2016-09-07 17:35:51 -07:00
Shaunak Kashyap 0a89f0b984 Merge branch 'master' into reporting/5.0/encrypt-headers
Original commit: elastic/x-pack-elasticsearch@38c755ce1e
2016-09-07 18:39:51 -05:00
jaymode 2257cc77f6 security: add reserved metadata to the ingest admin role
Closes elastic/elasticsearch#3376

Original commit: elastic/x-pack-elasticsearch@1909da2fb0
2016-09-07 19:19:02 -04:00
Joe Fleming f3b8182147 Merge pull request elastic/elasticsearch#3314 from w33ble/reporting-fix/use-absolute-times
Use absolute times for saved objects in reports

Original commit: elastic/x-pack-elasticsearch@2e510502ec
2016-09-07 16:11:37 -07:00
Joe Fleming 3ed10b073e Merge branch 'master' into reporting-fix/use-absolute-times
Original commit: elastic/x-pack-elasticsearch@176c3b72cc
2016-09-07 14:58:03 -07:00
Joe Fleming 0941589e53 Merge branch 'master' into reporting-fix/user-permissions
Original commit: elastic/x-pack-elasticsearch@df4e1e6ab5
2016-09-07 14:49:27 -07:00
Court Ewing 8bd16c0ee2 Merge pull request elastic/elasticsearch#3375 from epixa/3293-portselection
reporting: Improved phantom.js port selection

Original commit: elastic/x-pack-elasticsearch@2fdf3b3417
2016-09-07 17:43:18 -04:00
Joe Fleming 42833ddf55 Merge pull request elastic/elasticsearch#3377 from w33ble/reporting-fix/image-capture
Fix image capturing in Phantom

Original commit: elastic/x-pack-elasticsearch@0baf1a097d
2016-09-07 14:34:50 -07:00
Joe Fleming 4df7423ccb Merge branch 'master' into reporting-fix/use-absolute-times
Original commit: elastic/x-pack-elasticsearch@83684bb720
2016-09-07 14:10:20 -07:00
Lukas Olson 61e264b13d Merge pull request elastic/elasticsearch#3340 from lukasolson/fix/session-timeout-warning-system-api
Don't reset session timeout warning for system API calls

Original commit: elastic/x-pack-elasticsearch@6f66db1a74
2016-09-07 14:05:12 -07:00
Lukas Olson a1d8967d62 Merge branch 'master' into fix/session-timeout-warning-system-api
Original commit: elastic/x-pack-elasticsearch@13680956c4
2016-09-07 11:22:36 -07:00
Court Ewing 6d48cd5e23 Merge pull request elastic/elasticsearch#3338 from lukasolson/fix/remove-client-cookie
Don't store user information in cookie

Original commit: elastic/x-pack-elasticsearch@5a4c205e6b
2016-09-07 14:16:10 -04:00
Joe Fleming 38f87b59be Merge branch 'master' into reporting-fix/use-absolute-times
Original commit: elastic/x-pack-elasticsearch@bdc85eb9fb
2016-09-07 10:21:32 -07:00
Court Ewing aa7fc0aad3 Merge pull request elastic/elasticsearch#3343 from epixa/xsrfheader
Check new kbn-xsrf header in auth redirect

Original commit: elastic/x-pack-elasticsearch@3b74ae9e25
2016-09-07 13:01:53 -04:00
Alexander Reelsen 0f571685b9 Watcher: Fix proxy xcontent serialization (elastic/elasticsearch#3364)
Calling to xcontent in a HttpRequest, with a proxy enabled, lead to
serialization exceptions, resulting in failing to write the watch
history.

Closes elastic/elasticsearch#3334

Original commit: elastic/x-pack-elasticsearch@a04dff686c
2016-09-07 16:29:15 +02:00
Alexander Reelsen a296e31a7c Watcher: Ensure triggered watch is deleted on thread pool rejection (elastic/elasticsearch#3049)
This fixes a bug I found with a customer when he updated from 1.x to 2.x.
Due to an BWC incompatible change in the watch history mapping and a thread
pool rejection during execution a watch was not removed from the triggered
watches and tried to be executed again.

While trying to fix it it turned out that the execution of the failure
test case was still done in the transport thread and thus required some
offloading to another thread pool.

Original commit: elastic/x-pack-elasticsearch@df04ce31f2
2016-09-07 15:55:33 +02:00
Alexander Reelsen bb033f1e00 Watcher: Clear out WatchStore on watch index deletion (elastic/elasticsearch#2807)
If someone deletes the watch index (i.e. by deleting all indices), the watcher
in memory store still contains all the watches and tries to execute watches -
which results in exceptions as the watch itself cannot be updated anymore.

In order to minimize this problem (it cant be get rid of completely), we should
act accordingly if the watch index goes missing (either deleted or closed) and
clear out the memory representation of watches in the watchstore as well as trying
to finish all the current executions.

Closes elastic/elasticsearch#2794

Original commit: elastic/x-pack-elasticsearch@12d98cd566
2016-09-07 15:06:03 +02:00
Court Ewing f53fd02da0 Merge pull request elastic/elasticsearch#3305 from w33ble/reporting-fix/whitelisted-headers
Reporting fix/whitelisted headers

Original commit: elastic/x-pack-elasticsearch@2818f32098
2016-09-06 19:08:29 -04:00
Joe Fleming faf98b757c Merge pull request elastic/elasticsearch#3303 from w33ble/reporting-fix/wrap-json-parsing
Reporting fix/wrap json parsing

Original commit: elastic/x-pack-elasticsearch@2d2c3e8dbe
2016-09-06 15:43:33 -07:00
Joe Fleming 45dc0814d5 Merge pull request elastic/elasticsearch#3302 from w33ble/reporting-fix/date-pattern
Fix queue index pattern date

Original commit: elastic/x-pack-elasticsearch@9ab1f502d2
2016-09-06 15:41:58 -07:00
javanna ad5606d1f7 [TEST] don't use null script lang in WatcherUtilsTests
same as elastic/x-pack@9e1e0988c1 but for testSerializeSearchRequest this time.

Original commit: elastic/x-pack-elasticsearch@aa7e3814b6
2016-09-06 23:00:56 +02:00
Lukas Olson f48e4ecc95 Merge branch 'fix/remove-client-cookie' into fix/session-timeout-warning-system-api
Original commit: elastic/x-pack-elasticsearch@8588d10f93
2016-09-06 13:51:57 -07:00
Lukas Olson 528a61e372 Merge branch 'master' into fix/remove-client-cookie
Original commit: elastic/x-pack-elasticsearch@1768d14f1e
2016-09-06 13:51:38 -07:00
Lukas Olson 0036a7f620 Merge branch 'master' into fix/disabled-ui-select
Original commit: elastic/x-pack-elasticsearch@4d9812f2c2
2016-09-06 13:51:24 -07:00
Nik Everett edb8c12f75 Always use mustache language
Using a null language just defaults to painless which is wrong for
watcher.

Original commit: elastic/x-pack-elasticsearch@48bdc31695
2016-09-06 15:45:23 -04:00
Nik Everett 05222b0d3f Deal with ConcreteShardRequest
It exists now and the intercepters have to deal with it.

Original commit: elastic/x-pack-elasticsearch@fdef0578ed
2016-09-06 15:42:06 -04:00
Martijn van Groningen 36486a7993 Changes for PR: https://github.com/elastic/elasticsearch/pull/20310
Original commit: elastic/x-pack-elasticsearch@9db3d05274
2016-09-06 18:44:41 +02:00
jaymode 7965608add security: make control of logfile audit output consistent with index output
This change moves the logfile audit output from determining what to log based on the
logger level to a enum based configuration that is used by the index output.

A few notable changes were made:

* We alway log all the information we have except for the request body
* The request body is no longer logged by default for REST events; the user needs to
explicitly opt in as there could be sensitive data in the body
* Added a `realm_authentication_failed` event that separates overall authentication
failure from that of an individual realm

Original commit: elastic/x-pack-elasticsearch@343a2bcdd9
2016-09-06 10:50:33 -04:00
Nik Everett 89ce4ebb08 Fix remaining fractional TimeValue issues with watcher
Removes the remaining spots where watcher makes fractional TimeValues.

Closes elastic/elasticsearch#3231

Original commit: elastic/x-pack-elasticsearch@22b0d37ed3
2016-09-06 09:53:22 -04:00
jaymode cd9add5350 test: remove unnecessary verify check
Original commit: elastic/x-pack-elasticsearch@f748c0a565
2016-09-06 08:36:04 -04:00
jaymode 1e6a924e74 security: add support for disabling users
This change adds support for disabling users. Users can be disabled by setting the enabled
property to false and the AuthenticationService will check to make sure that the user is enabled.
If the user is not enabled, this will be audited as an authentication failure.

Also as part of this work, the AnonymousUser was cleaned up to remove having a static instance
that caused issues with tests.

Finally, the poller of users was removed to simplify the code in the NativeUsersStore. In our other
realms we rely on the clear cache APIs and the timeout of the user cache. We should have the
same semantics for the native realm.

Closes elastic/elasticsearch#2172

Original commit: elastic/x-pack-elasticsearch@0820e40183
2016-09-06 07:41:48 -04:00
Chris Earle ec008ec4a9 Looong lines
Original commit: elastic/x-pack-elasticsearch@8c7931c179
2016-09-05 19:05:28 -04:00