Commit Graph

4416 Commits

Author SHA1 Message Date
Alexander Reelsen 1ba5f8fb30 Watcher: Enable with STANDARD license (elastic/elasticsearch#4838)
In order to have monitoring use alerts on cloud, we need to
enable watcher when the STANDARD license is in place. For
more information, please refer to the referenced issue.

Closes elastic/elasticsearch#4766

Original commit: elastic/x-pack-elasticsearch@9d5547274c
2017-02-08 17:38:00 +01:00
Simon Willnauer 0e779b41de Followup from elastic/elasticsearchelastic/elasticsearch#23042 (elastic/elasticsearch#4895)
This change accomodates for the renamings done in elastic/elasticsearchelastic/elasticsearch#23042

Original commit: elastic/x-pack-elasticsearch@c290c8ecc4
2017-02-08 14:40:17 +01:00
Tim Vernum 734a4ee66d Prevent default passwords in production mode (elastic/elasticsearch#4724)
Adds a new `xpack.security.authc.accept_default_password` setting that defaults to `true`. If it is set to false, then the default password is not accepted in the reserved realm.

Adds a bootstrap check that the above setting must be set to `false` if security is enabled.  

Adds docs for the new setting and bootstrap.

Changed `/_enable` and `/_disable`, to store a blank password if the user record did not previously exist, which is interpreted to mean "treat this user as having the default password". The previous functionality would explicitly set the user's password to `changeme`, which would then prevent the new configuration setting from doing its job.

For any existing reserved users that had their password set to `changeme`, migrates them to the blank password (per above paragraph)

Closes: elastic/elasticsearch#4333

Original commit: elastic/x-pack-elasticsearch@db64564093
2017-02-08 16:19:55 +11:00
Boaz Leskes aadbe81767 TransportService.connectToNode should validate remote node ID (elastic/elasticsearch#4866)
companion PR for elastic/elasticsearchelastic/elasticsearch#22828

Original commit: elastic/x-pack-elasticsearch@2465a03ebc
2017-02-07 22:11:54 +02:00
Tim Brooks b59ead91e0 Remove usages for forbidden apis or suppress (elastic/elasticsearch#4850)
This is related to elastic/elasticsearchelastic/elasticsearch#22964. Methods that could
open socket connections are being made forbidden apis. This commit
either replaces usages with URL.openStream() with a call to
FileSystemUtils.openFileURLStream(URL url) (in the case of a file url)
or adds SuppressForbidden annotations.

Original commit: elastic/x-pack-elasticsearch@93b1b11375
2017-02-07 12:38:21 -06:00
Jay Modi bd04b30acd security: transport always uses TLS (elastic/elasticsearch#4738)
This commit brings back support an auto-generated certificate and private key for
transport traffic. The auto-generated certificate and key can only be used in development
mode; when moving to production a key and certificate must be provided.

For the edge case of a user not wanting to encrypt their traffic, the user can set
the cipher_suites setting to `TLS_RSA_WITH_NULL_SHA256` or a like cipher, but a key/cert
is still required.

Closes elastic/elasticsearch#4332

Original commit: elastic/x-pack-elasticsearch@b7a1e629f5
2017-02-07 11:39:31 -05:00
Tim Vernum 8820902f0c [ci] Print git revision of elasticsearch to ci log (elastic/elasticsearch#4782)
The output `Checked out Elasticsearch revision: ...` shows which revision of elasticsearch was included in the CI build. This makes it easier to diagnose and resolve x-pack build failures.

Original commit: elastic/x-pack-elasticsearch@8c39aafc2d
2017-02-07 10:36:49 +11:00
Tim Vernum 286d62f00e Don't clear realm-cache during security upgrade on basic license (elastic/elasticsearch#4806)
If a user has a basic license, but previously had a full-featured license (e.g. a trial license that expired) then they may have .security index that needs to be migrated even though security is not allowed under their license.

This change makes the clearing of the realm-cache conditional on the license state. If X-Pack is running on a license that does not allow auth, then, when the `logstash_system` user is disabled as part of an upgrade migration, the cache is not cleared.

This change also fix a bug whereby a mapping update could take place even if a data migration was in progress, which could cause the `logstash_system` user to be temporarily enabled when it ought not be.

Original commit: elastic/x-pack-elasticsearch@f272e2b19f
2017-02-07 08:56:04 +11:00
Alexander Reelsen 752356916d Tests: Fix failing test
Caused by removing search QUERY_AND_FETCH

Original commit: elastic/x-pack-elasticsearch@18070b40bb
2017-02-06 19:40:15 +01:00
Simon Willnauer 14e97565fa Remove obsolet search type QUERY_AND_FETCH
Original commit: elastic/x-pack-elasticsearch@10329462bd
2017-02-06 18:26:01 +01:00
Boaz Leskes ccf3ac9c82 fix compilation issue caused by https://github.com/elastic/elasticsearch/pull/22984
Original commit: elastic/x-pack-elasticsearch@6e4d902deb
2017-02-06 16:48:44 +01:00
Suyog Rao 77c477f575 [Monitoring UI] Add Logstash persistent queue stats (elastic/elasticsearch#4775)
* Add persistent queue charts to Logstash "Advanced" view.

Original commit: elastic/x-pack-elasticsearch@62fac9f469
2017-02-06 16:56:10 +05:30
Jason Tedor 65ddba8585 Avoid certgen status logger error message
Today when running the certgen command, an error message is output by
the status logger. This is due to a logger instance being created before
logging is configured. This happens because the class initializer for
CertificateTool runs the class initializer for ParseField which creates
a deprecation logger which creates a logger. This commit fixes this
issue by wrapping the parser in another class so that we can defer class
initialization until it's actually needed, thus deferring creating the
logger instance until after logging is initialized.

Relates elastic/elasticsearch#4831

Original commit: elastic/x-pack-elasticsearch@00f978c878
2017-02-04 16:07:36 -05:00
Tim Brooks 6675f6bc8d Add missing doPrivileged blocks for connect ops (elastic/elasticsearch#4815)
This is related to elastic/elasticsearchelastic/elasticsearch#22116. There were a few
places where x-pack opens socket connects that were missed in PR elastic/elasticsearch#4701.
This commit adds the doPrivileged blocks.

Original commit: elastic/x-pack-elasticsearch@32bfa087f0
2017-02-03 14:48:07 -06:00
Jason Tedor 45734aae22 Upgrade checkstyle to version 7.5
This commit upgrades the checkstyle configuration from version 5.9 to
version 7.5, the latest version as of today. The main enhancement
obtained via this upgrade is better detection of redundant modifiers.

Relates elastic/elasticsearch#4810

Original commit: elastic/x-pack-elasticsearch@2c9b7d23dc
2017-02-03 09:47:04 -05:00
Jason Tedor 02bcd9510f Enable disabled rolling upgrade tests
These tests were disabled due to a bug introduced when
sequence-number-based recovery was introduced. This commit enables these
tests again.

Relates elastic/elasticsearch#4801

Original commit: elastic/x-pack-elasticsearch@e555bfec1a
2017-02-03 08:12:52 -05:00
Adrien Grand 16be5ecc81 Clean up field-level security. (elastic/elasticsearch#4784)
This commit reuses the automaton that defines the allowed fields in
`FieldSubsetReader` rather than resolving the list of all matching fields from
the mapping. As a side-effect this change solves a bug that unmapped fields
could previously not be read from the source. Moreover it avoids determinization
errors in the case that the number of matching fields is high.

It also uses `CharacterRunAutomaton` to evaluate automata against a given
string, which should be faster than naively stepping into the automaton since
`CharacterRunAutomaton` builds a lookup table of transitions.

Closes elastic/elasticsearch#4679

Original commit: elastic/x-pack-elasticsearch@a30913dbd5
2017-02-03 12:31:04 +01:00
Alexander Reelsen 44618b5b87 Watcher: Prevent NPE if watcher indices are closed (elastic/elasticsearch#4763)
The way we check for the triggered watches on start-up did not take into account
that an index could be closed and thus resulted in an NPE.

This commit adds a check to ensure that the watch index and triggered watches index
are open, before trying to check if all primary shards are active.

Original commit: elastic/x-pack-elasticsearch@ee05779963
2017-02-03 11:44:48 +01:00
Jay Modi 1f32ef21a2 Update to pass content type from the rest APIs to transport requests (elastic/elasticsearch#4689)
This change adapts x-pack to pass on the parsed XContentType from rest requests to transport
requests and use this value in place of attempting to auto-detect the content type.

Original commit: elastic/x-pack-elasticsearch@57475fd403
2017-02-02 14:08:34 -05:00
Ali Beyad 8cab4fec4b [TEST] adds AwaitsFix to failing test
Original commit: elastic/x-pack-elasticsearch@9e8f6ce047
2017-01-31 22:50:17 -05:00
Ali Beyad 4f4d262323 Adds bwc index for 5.2.0
Original commit: elastic/x-pack-elasticsearch@7b526b241a
2017-01-31 17:13:01 -05:00
Jack Conradson 7f0ecc4b30 Change Namespace for Stored Script to Only Use Id (elastic/elasticsearch#4387)
Changes the behavior in x-pack necessary to support the elasticsearch change elastic/elasticsearch#22206.

Original commit: elastic/x-pack-elasticsearch@916e72e263
2017-01-31 13:28:03 -08:00
Peter Dyson 81171bb6ff Merge pull request elastic/elasticsearch#4601 from elastic/geekpete-patch-1
[DOCS] [Watcher] Add additional detail around auth and settings

Original commit: elastic/x-pack-elasticsearch@ce22ef71e2
2017-01-30 09:54:54 +10:00
Nik Everett cdfa20b83f Handle moving some classes from reindex to core
Original commit: elastic/x-pack-elasticsearch@cdf98a70f7
2017-01-27 16:08:32 -05:00
Simon Willnauer 795a8a84ed Remove obsolete transport handlers
Original commit: elastic/x-pack-elasticsearch@9a9ce85b76
2017-01-27 16:05:47 +01:00
Tim Brooks 5f84ddee7c Add doPrivileged blocks for socket connect ops (elastic/elasticsearch#4701)
This is related to elastic/elasticsearch#22116. X-pack opens socket connections for a number
of pieces of functionality (Active Directory support, ssl, email
notification, etc). As SocketPermissions are transitioned out of core,
x-pack will require connect permission. This pull request wraps
operations requiring these permissions in doPrivileged blocks.

Original commit: elastic/x-pack-elasticsearch@1bfee97550
2017-01-26 13:18:36 -06:00
Nik Everett ae994c72d3 Fix compilation in Eclipse (elastic/elasticsearch#4745)
Eclipse compiler is silly....

Original commit: elastic/x-pack-elasticsearch@1c7d19bffe
2017-01-26 12:00:52 -05:00
Suyog Rao 31bda61d92 [Monitoring UI] Add cgroups charts to LS advanced view
* Adds Advanced view for Logstash
* Adds 2 cgroups stats based chart in advanced view

Fixes elastic/elasticsearch#4386

Original commit: elastic/x-pack-elasticsearch@453847bcd3
2017-01-26 08:28:49 -08:00
Stacey Gammon 48d01dad4c Merge pull request elastic/elasticsearch#4736 from Stacey-Gammon/remove-safe-confirm-references
Switch from safeConfirm to confirmModal

Original commit: elastic/x-pack-elasticsearch@feddbd00a2
2017-01-25 17:45:19 -05:00
Simon Willnauer 133591a26f [TEST] Remove dfs_query_and_fetch - this mode is internal only and will be removed
Original commit: elastic/x-pack-elasticsearch@f6bb9ce498
2017-01-25 20:18:27 +01:00
Chris Earle 9fad3cf85c [Watcher] Allow Index Action to set _id (elastic/elasticsearch#4694)
This adds a "doc_id" parameter to the index action itself, which can accept a single ID value. This also allows the payload to set _id (or _doc._id) for each document being indexed in order to support this with bulk index actions.

If doc_id and _id are used together, then this blocks the action.

Original commit: elastic/x-pack-elasticsearch@f186ccceb8
2017-01-25 13:03:26 -05:00
Alexander Reelsen 9d002430b5 Tests: Remove class only used by tests
The `Integers` class was only used in tests (but lurked around in the src) and is not needed.
Also replaced some lambda calls with their shorter equivalents.

Original commit: elastic/x-pack-elasticsearch@a81a5c33d3
2017-01-24 18:10:05 +01:00
Alexander Reelsen 984b1b0dd1 Watcher: Add dedicated acknowledged state (elastic/elasticsearch#4588)
In order to display a better UI page for the watch history, the state
of a throttled watch should be visualized. However, right now there
is no way to differentiate between a time throttled watch and a user
acknowledged watch (or action). This commit introduces a new type in a
throttled result, which in turn is used to set the execution state of
a watch.

Closes elastic/elasticsearch#4531

Original commit: elastic/x-pack-elasticsearch@b86e666e54
2017-01-24 16:33:29 +01:00
Luca Cavanna 5e8dd26d93 adapt to "move es. headers to metadata set in ElasticsearchException and stop returning them as response headers" (elastic/elasticsearch#4693)
Original commit: elastic/x-pack-elasticsearch@91abdf73c8
2017-01-24 16:13:03 +01:00
javanna a8bb433aa1 fix compile errors due to upstream changes
Original commit: elastic/x-pack-elasticsearch@40a2561fa3
2017-01-24 13:09:01 +01:00
Simon Willnauer 1998b7ef46 Remove RequestContext from Security (elastic/elasticsearch#4710)
RequestContext is a leftover from when we had no thread context. This
commit removes the last place where it was used and uses the thread context
instead.

Original commit: elastic/x-pack-elasticsearch@50a2bff400
2017-01-23 15:44:40 +01:00
Simon Willnauer 8651c0ad9f Log exceptions in onFailure instead of bubbling up (elastic/elasticsearch#4709)
Today we wrap the checked IOException in an unchecked exception when
sending back a failure the security transport interceptor. Yet, if that failure handling
in-turn fails due to a broken response channel we should rather log the exception instead
of bubbling it up since it can have unforeseeable side-effects.

Relates to elastic/elasticsearch#4706

* fix line len

Original commit: elastic/x-pack-elasticsearch@03c3826112
2017-01-23 15:08:48 +01:00
Simon Willnauer b6703c1515 Preserve `forceExecution` flag when forking off handler threads after authentication (elastic/elasticsearch#4706)
Today we might get a rejection on a critical operation where `forceExecution=true` but
due to the fact that the forceExecution flag is not passed to the transport interceptor
interface we were not able to preserve this flag when forking off the request after authentication.
This causes serious issues if for instance a replication handler is rejected.

Closes elastic/elasticsearch#4704

Original commit: elastic/x-pack-elasticsearch@f0aad7dede
2017-01-23 11:07:36 +01:00
Jay Modi 9005e9fdb9 security: filter content of known requests with passwords (elastic/elasticsearch#4700)
This commit adds a mechanism for defining known sensitive values in rest bodies so that
these can be filtered when auditing the request body.

Original commit: elastic/x-pack-elasticsearch@d138a6bff7
2017-01-20 14:05:23 -05:00
Nik Everett d690c5f789 Deguice rest handlers (elastic/elasticsearch#4598)
Since some of the rest handlers depend on components constructed
in `createComponents` we use `SetOnce` to save a reference to
the components at the class level and reuse the reference in
`initRestHandlers`. This does require that `initRestHandlers`
is called after `createComponents` but I think that is fairly
reasonable.

Original commit: elastic/x-pack-elasticsearch@4fd87ad911
2017-01-20 11:49:20 -05:00
Yannick Welsch 8f70653233 Close XContentParser when executing searches from Watcher (elastic/elasticsearch#4696)
Also removes unused method convertToObject from XContentUtils that does not close XContentParser either.

Original commit: elastic/x-pack-elasticsearch@99ce977c55
2017-01-20 12:40:48 +01:00
Alexander Reelsen 84936d57ad Watcher: Fix XContentFilterKeysUtils when using nested lists (elastic/elasticsearch#4635)
The ability to filter JSON inside of arrays was stopping after the first element was found.

Closes elastic/elasticsearch#4614

Original commit: elastic/x-pack-elasticsearch@452cf1c49d
2017-01-20 12:22:20 +01:00
Alexander Reelsen 8011912ce7 Tests: Added some tests for proper 404 responses on watch actions (elastic/elasticsearch#4620)
Relates elastic/elasticsearch#4616 elastic/elasticsearch#4617

Original commit: elastic/x-pack-elasticsearch@7f73792507
2017-01-20 12:01:44 +01:00
Boaz Leskes 630b5fd836 Move ContextPreservingActionListener to core (elastic/elasticsearch#4692)
Original commit: elastic/x-pack-elasticsearch@c3e5762ffc
2017-01-20 10:14:34 +01:00
Daniel Mitterdorfer 6ed83cc8ea Merge pull request elastic/elasticsearch#4380 from danielmitterdorfer/strict-booleans
This PR removes all leniency in the conversion of Strings to booleans: "true" is converted to the boolean value true, "false" is converted to the boolean value false. Everything else raises an error.

Relates elastic/elasticsearchelastic/elasticsearch#22200

Original commit: elastic/x-pack-elasticsearch@a505df1f5d
2017-01-19 08:00:06 +01:00
Ali Beyad 5aae30f722 [TEST] fixes known handler test
Original commit: elastic/x-pack-elasticsearch@98a8f23d23
2017-01-18 16:42:08 -05:00
Ali Beyad db129051c8 [TEST] fix test to use the stored field instead of id for
stored_template_role

Original commit: elastic/x-pack-elasticsearch@18c9a78587
2017-01-18 12:34:12 -05:00
Ali Beyad 159bf63cd4 [TEST] fix security's request handler tests to check in the
knownHandlers set, not the knownActions

Original commit: elastic/x-pack-elasticsearch@63f3744853
2017-01-18 12:19:42 -05:00
Simon Willnauer 5b5c77f573 Follow up for elastic/elasticsearchelastic/elasticsearch#22677 (elastic/elasticsearch#4670)
This commit picks up some improvments from elastic/elasticsearch#22677 that streamlines correct restore of stored contexts.

Original commit: elastic/x-pack-elasticsearch@0259de50c8
2017-01-18 16:18:10 +01:00
Daniel Mitterdorfer 226d6872ff Merge remote-tracking branch 'origin/master' into strict-booleans
Original commit: elastic/x-pack-elasticsearch@a81d65f77f
2017-01-18 13:39:52 +01:00