Commit Graph

68 Commits

Author SHA1 Message Date
Lee Hinman affc9e3563 Move more XContent.createParser calls to non-deprecated version (elastic/x-pack-elasticsearch#3928)
Part 2

This moves more of the callers to pass in the DeprecationHandler.

Relates to elastic/x-pack-elasticsearch#28504

Original commit: elastic/x-pack-elasticsearch@e0b52bf050
2018-02-14 09:41:26 -07:00
Lee Hinman e8ef20b219 Move more XContent.createParser calls to non-deprecated version (elastic/x-pack-elasticsearch#3927)
* Move more XContent.createParser calls to non-deprecated version

This moves more of the callers to pass in the DeprecationHandler.

Relates to https://github.com/elastic/elasticsearch/issues/28504

* Mockito ಠ_ಠ

Original commit: elastic/x-pack-elasticsearch@81cb99ba52
2018-02-14 09:24:47 -07:00
Yogesh Gaikwad 20503a997f X-Pack-Security: Improve error messaging during setup password (elastic/x-pack-elasticsearch#3809)
* X-Pack-Security: Improve error messaging during setup password

Changes are done to improve logging and exception messages when
setup passwords. Now it checks whether
- X-Pack plugin is available
- X-Pack Security is available 
- X-Pack Security is enabled
by invoking /_xpack query and throws a specific error message.
1. Modifications to simple http client to return http response.
2. Changes in SetupPasswordTool for throwing appropriate exceptions.
2. Unit tests

relates elastic/x-pack-elasticsearch#3476

Original commit: elastic/x-pack-elasticsearch@eb0c5a19c5
2018-02-14 13:11:00 +11:00
Tim Brooks 5dbbe8fef8 Default to basic license at startup (elastic/x-pack-elasticsearch#3878)
This is related to elastic/x-pack-elasticsearch#3877. This commit modifies the license settings to
default to self generating a basic license.

Original commit: elastic/x-pack-elasticsearch@cd6ee8e06f
2018-02-12 12:57:04 -07:00
Ryan Ernst 4e578fe7cc Build: Replace provided configuration with compileOnly (elastic/x-pack-elasticsearch#3868)
This is the xpack side of elastic/elasticsearch#28564

Original commit: elastic/x-pack-elasticsearch@60033824a4
2018-02-09 11:30:43 -08:00
Ioannis Kakavas 7504e33da7 Change value of METADATA_REFRESH for the tests (elastic/x-pack-elasticsearch#3879)
Ensure that the next triggered metadata refresh is always in the
future. relates elastic/x-pack-elasticsearch#3815

Original commit: elastic/x-pack-elasticsearch@edd9ba911d
2018-02-09 18:22:26 +01:00
Albert Zaharovits fb8adb4a1c [Security][Tests] Azeri(Turkish) locale tripps opensaml dependency
Original commit: elastic/x-pack-elasticsearch@9c4ecabe48
2018-02-09 13:57:09 +02:00
Albert Zaharovits a57999e1e1 [Security][Tests] Print LDAP reconnect stacktrace on fail
Original commit: elastic/x-pack-elasticsearch@18f235197c
2018-02-08 13:41:02 +02:00
Alexander Reelsen 064a0819d9 Cleanup: Remove HaltedClock (elastic/x-pack-elasticsearch#3731)
java.time features it's own halted clock, called a fixed clock, we can
use that one.

On top of that the watcher xcontent parser does not need a clock at all,
just a timestamp when parsing happened.

Original commit: elastic/x-pack-elasticsearch@2061aeffe1
2018-02-08 09:24:08 +01:00
Tim Vernum 3102b94946 [TEST] Don't fail IndexAuditTrailTests if the test runs very fast
Relates: elastic/x-pack-elasticsearch#3472

Original commit: elastic/x-pack-elasticsearch@00fde8973f
2018-02-08 16:44:58 +11:00
Ryan Ernst f360d2d8d7 Build: Cleanup gradle files (elastic/x-pack-elasticsearch#3867)
There were a number of leftover unnecessary elements in the module
build.gradle files that were holdovers from copying the original plugin
build.gradle. This commit removes these elements.

Original commit: elastic/x-pack-elasticsearch@08babbd520
2018-02-07 16:58:09 -08:00
Nik Everett b95745956e Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@5f6bbb588a
2018-02-07 09:38:13 -05:00
Tim Vernum 7bb2cba14f [Security] Reset IndexAuditTrail to INITIALISED before start (elastic/x-pack-elasticsearch#3807)
Calling start() when already in the STARTING state doesn't do anything, so the component
gets stuck in STARTING state forever.

Also: wait on the required index name not just the cluster.

Also: added more logging to help diagnose such issues (either in RemoteIndexAuditTrailStartingTests or production)

Original commit: elastic/x-pack-elasticsearch@fb81214fe7
2018-02-07 12:30:23 +11:00
Nik Everett 5b46899cc3 Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@9a49ecbbd5
2018-02-06 14:40:55 -05:00
Jay Modi 18c04c714d [Security] handle null values when attempting to parse as DN when mapping roles (elastic/x-pack-elasticsearch#3791)
This commit adds special handling for null values when building a DistinguishedNamePredicate for
role mapping. Previously this would have resulted in an exception from the unboundid ldapsdk as
the DN is invalid.

relates elastic/x-pack-elasticsearch#3787

Original commit: elastic/x-pack-elasticsearch@9386dae03e
2018-02-06 10:19:49 -07:00
Costin Leau b5dd4c649a SQL: Minor fixes
Original commit: elastic/x-pack-elasticsearch@14ea747e20
2018-02-02 17:50:33 +02:00
Nik Everett f66e01369a Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@6ba2651f46
2018-02-01 17:21:37 -05:00
Tim Vernum 415bb7f039 Clear Realm Caches on role mapping health change (elastic/x-pack-elasticsearch#3782)
If any of the follow take place on security index, then any cached role mappings
are potentially invalid and the associated realms need to clear any cached users.
- Index recovers from red
- Index is deleted
- Index becomes out-of-date / not-out-of-date

Original commit: elastic/x-pack-elasticsearch@1bcd86fcd4
2018-02-01 10:01:57 +11:00
Lee Hinman 1b36133988 Merge remote-tracking branch 'origin/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@4176773659
2018-01-31 10:49:25 -07:00
Tim Vernum d0f6d273d0 [TEST] Add additional logging to test
Attempt to diagnose elastic/x-pack-elasticsearch#3292 if it occurs again.

Original commit: elastic/x-pack-elasticsearch@eb262540f0
2018-01-31 18:26:57 +11:00
Simon Willnauer 570411c2dc Remove all tribe related code, comments and documentation (elastic/x-pack-elasticsearch#3784)
Relates to elastic/elasticsearch#28443

Original commit: elastic/x-pack-elasticsearch@5c4e7fccc7
2018-01-30 20:40:46 +01:00
Nik Everett 6631fe0376 Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@ec3a82494e
2018-01-30 12:50:27 -05:00
Jay Modi 71788671ad Test: fix skip check for EllipticCurveSSLTests (elastic/x-pack-elasticsearch#3771)
This change fixes the skip check for the EllipticCurveSSLTests. The skip check that is in the test
was added to proctect against failures on JVMs that do not support EC ciphers such as the packaged
openjdk in some linux distributions. The old skip check did not execute until the cluster was up
so the test would still fail with errors such as no cipher suites in common. This change moves the
check into a before class method that checks availability.

Original commit: elastic/x-pack-elasticsearch@5550ca4e1e
2018-01-30 08:21:15 -07:00
David Kyle 10fae5e58f [ML][TEST] Scheduled events integration test (elastic/x-pack-elasticsearch#3555)
* Scheduled events integration test

Original commit: elastic/x-pack-elasticsearch@8181645fe5
2018-01-30 10:38:13 +00:00
Tim Vernum da7560a079 [Security] Correct DN matches in role-mapping rules (elastic/x-pack-elasticsearch#3704)
Changes the behaviour of the role mapping API to perform a "DistinguishedNameMatch"
when the field is a DN. This is achieved by moving the responsibility for defining
the matching rules from the expression to the data (ExpressionModel)

Because the role mapping API is used within the SAML realm, which may or may not be
using DNs, this implementation assumes that the "dn" and "groups" should be
compared as DNs if they parse as a DN.
For SAML this behaviour will generally do the right thing, as members of the "groups"
field might be DNs (if the data is sourced from an LDAP directory) but often will not be.

Original commit: elastic/x-pack-elasticsearch@3a4dfbba79
2018-01-30 15:47:17 +11:00
Lee Hinman dc6b5ce453 [BUILD] have security's build.gradle use xpackModule for SQL dep
Original commit: elastic/x-pack-elasticsearch@88027b1d3b
2018-01-29 15:51:55 -07:00
Lee Hinman a789944b8d Merge remote-tracking branch 'origin/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@f96f80f0d0
2018-01-29 15:42:01 -07:00
jaymode 4271fd7cc3 Test: use System.lineseperator instead of `\n` in SamlSpMetadataBuilderTests
This commit fixes SamlSpMetadataBuilderTests#testBuildFullMetadata failures on windows due to
differing newline endings.

Original commit: elastic/x-pack-elasticsearch@3181c96e46
2018-01-29 13:47:42 -07:00
Simon Willnauer 32d4e7f9d0 [TEST] Use `ensureYellowAndNoInitializingShards` in monitoring test (elastic/x-pack-elasticsearch#3768)
Monitoring creates indices concurrently to the test execution. In that
case ensureYellow might not be enough and will cause test-failures when shards
are not active etc. This change uses a new method added in core to ensure shards
are not initializing anymore.

relates elastic/x-pack-elasticsearch#2672
Relates to elastic/elasticsearch#28416

Original commit: elastic/x-pack-elasticsearch@661e87f2ee
2018-01-29 20:46:35 +01:00
Costin Leau 07658cc04f SQL: replace JDBC transport meta columns with SQL call (elastic/x-pack-elasticsearch#3740)
Replace meta column endpoint with SYS COLUMNS command

Original commit: elastic/x-pack-elasticsearch@819874bc5b
2018-01-29 19:18:24 +02:00
Simon Willnauer e7b2102126 Remove extension executables
Relates to elastic/x-pack-elasticsearch#3734

Original commit: elastic/x-pack-elasticsearch@da75402bc9
2018-01-29 10:52:27 +01:00
Costin Leau e929d16d76 SQL: Replace ListTable transport with SQL command (elastic/x-pack-elasticsearch#3675)
Introduce system commands as alternative to meta HTTP endpoints
Pass in cluster name
Use 'BASE TABLE' instead of 'INDEX' when describing a table to stick
with the SQL terminology

Original commit: elastic/x-pack-elasticsearch@600312b8f7
2018-01-27 11:54:19 +02:00
Ryan Ernst 5a86450df7 Build: Replace references to x-pack-elasticsearch paths with helper methods (elastic/x-pack-elasticsearch#3748)
In order to more easily integrate xpack once it moves into the
elasticsearch repo, references to the existing x-pack-elasticsearch need
to be reduced. This commit introduces a few helper "methods" available
to any project within xpack (through gradle project extension
properties, as closures). All refeerences to project paths now use these
helper methods, except for those pertaining to bwc, which will be
handled in a followup.

Original commit: elastic/x-pack-elasticsearch@850668744c
2018-01-26 21:48:30 -08:00
Simon Willnauer 730e7075ab Remove XPackExtension in favor of SecurityExtensions (elastic/x-pack-elasticsearch#3734)
This change removes the XPackExtension mechanism in favor of
SecurityExtension that can be loaded via SPI and doesn't need
another (duplicate) plugin infrastructure

Original commit: elastic/x-pack-elasticsearch@f39e62a040
2018-01-26 16:14:11 +01:00
Jay Modi f07a850ba2 [SAML] add security permission to get the classloader (elastic/x-pack-elasticsearch#3720)
This adds the security permission to get the thread context classloader to the security
plugin after it was inadvertently removed in elastic/x-pack-elasticsearch#3651. This was not caught as there is a
dependency on monitoring for security and monitoring has the getClassLoader permission.

relates elastic/x-pack-elasticsearch#3719

Original commit: elastic/x-pack-elasticsearch@61ad950f5b
2018-01-25 13:01:48 -07:00
Michael Basnight e59bcd8f8e Enable the licenseHeaders task for plugins (elastic/x-pack-elasticsearch#3673)
Original commit: elastic/x-pack-elasticsearch@c9949b8ca5
2018-01-25 11:34:02 -06:00
Ryan Ernst 3e24ea3ed2 Test: Add internalClusterTest to xpack modules (elastic/x-pack-elasticsearch#3730)
This commit reenables running ITs in xpack by adding an internalClusterTest to xpack modules that contain ESIntegTestCase tests. The new task allows us to run these independently of rest integ tests, which are disabled for xpack modules because installing the bundled plugins directly is not quite the same as installing via the meta plugin. Some tests (ML) are moved to their own qa module to accommodate the need for a real cluster. A couple tests (monitoring and upgrade) have been marked as AwaitsFix.

Commits that have been folded into this commit:
* Move ML IT tests to qa/ml-native-tests
* Add internalClusterTest task and disable rest integ tests for xpack
modules. Also tweak ML tests and get upgrade tests working
* Adding the keystore and security back to the ml native tests
* Fixing native integ test
* Fix last ML test, add awaits fix to monitoring and upgrade tests
* cleanup PR
* fix checkstyle

Original commit: elastic/x-pack-elasticsearch@3c0ed6fd3b
2018-01-25 10:38:34 +01:00
Jay Modi 3baf6f3a7a Remove production from the message about license installation without TLS (elastic/x-pack-elasticsearch#3666)
This change simply changes the wording of the message that is returned to the user when
installation of a license is attempted with security enabled and TLS disabled. The term
"production" has been removed as it means something different to users.

See elastic/x-pack-elasticsearch#2636

Original commit: elastic/x-pack-elasticsearch@9739c72d66
2018-01-24 12:44:25 -07:00
Jay Modi 0baa45d9b3 [SAML] Find all tokens for a realm, not just the first 10 (elastic/x-pack-elasticsearch#3689)
This commit changes the token service to use a scroll based approach when finding all tokens by
the realm. Without this, we may only find a few tokens and leave some active that need to be
invalidated.

relates elastic/x-pack-elasticsearch#3688

Original commit: elastic/x-pack-elasticsearch@20e97b6aae
2018-01-24 11:07:51 -07:00
Costin Leau 3f4c789993 Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@348f0468b0
2018-01-24 19:19:51 +02:00
Jason Tedor e385b7dab4 Elevate privileges fetching metadata for SAML
We have to elevate privileges here as these invocations happen in a run
loop that will not have the correct privileges for socket connections.

Relates elastic/x-pack-elasticsearch#3671

Original commit: elastic/x-pack-elasticsearch@eab9f47583
2018-01-24 08:59:01 -05:00
Jason Tedor 3932635f98 Simplify security manager permissions
The security manager permissions were copied wholesale from pre-split
X-Pack. However, this grants unnecessary permissions to the child
plugins. This commit is a simple attempt at removing permissions that
are not needed in the child plugins.

Relates elastic/x-pack-elasticsearch#3651

Original commit: elastic/x-pack-elasticsearch@8325ed83d7
2018-01-24 08:57:54 -05:00
Michael Basnight 4475b718a4 Remove invalid use of XPackPlugin class (elastic/x-pack-elasticsearch#3700)
Original commit: elastic/x-pack-elasticsearch@f4aabe1259
2018-01-23 21:11:24 -06:00
Igor Motov de15f7123b Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@bbabfb1328
2018-01-23 15:41:59 -05:00
Jason Tedor c0790d6a49 Move x-pack-core to core package (elastic/x-pack-elasticsearch#3678)
This commit moves the source file in x-pack-core to a org.elasticsearch.xpack.core package. This is to prevent issues where we have compile-time success reaching through packages that will cross module boundaries at runtime (due to being in different classloaders). By moving these to a separate package, we have compile-time safety. Follow-ups can consider build time checking that only this package is defined in x-pack-core, or sealing x-pack-core until modules arrive for us.

Original commit: elastic/x-pack-elasticsearch@232e156e0e
2018-01-23 12:43:58 -06:00
Igor Motov 2be9cabbe9 Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@d6451b0f6b
2018-01-23 12:53:13 -05:00
Alexander Reelsen d065b087ee Revert "Cleanup: Remove HaltedClock (elastic/x-pack-elasticsearch#3664)"
This reverts commit elastic/x-pack-elasticsearch@f91c401a60 due to
failing tests, like

./gradlew :x-pack-elasticsearch:plugin:watcher:test -Dtests.seed=AE30350FCE96D26D -Dtests.class=org.elasticsearch.xpack.watcher.watch.WatchTests -Dtests.method="testParserSelfGenerated" -Dtests.security.manager=true -Dtests.locale=ja-JP -Dtests.timezone=EET

Original commit: elastic/x-pack-elasticsearch@e45d79d643
2018-01-23 18:08:54 +01:00
Alexander Reelsen e3da8fa4ae Cleanup: Remove HaltedClock (elastic/x-pack-elasticsearch#3664)
The HaltedClock was a leftover from moving over from our own Clock
implementation to a java.time one. java.time already has a fixed clock,
this one is not needed.

Original commit: elastic/x-pack-elasticsearch@f91c401a60
2018-01-23 14:13:22 +01:00
Simon Willnauer c70c1c7e98 Expose XPackExtensions via SPI (elastic/x-pack-elasticsearch#3530)
This change adds SPI loading for XPackExtensions that allows to extend
XPack via an ordinary plugin. This can co-exist with the existin
extension mechanism for the time being.

Original commit: elastic/x-pack-elasticsearch@bf02b56dee
2018-01-23 13:05:39 +01:00
Ryan Ernst 9d87b63ca4 Build: Fix third party audit task for xpack core (elastic/x-pack-elasticsearch#3656)
This commit re-enables thirdPartyAudit for x-pack core. Previously, when
xpack was a single plugin, it transitively picked up httpcore-nio
through the elasticsearch rest client. Now that xpack core does not
depend on the rest client, httpcore-nio must be added as a dependency.
Additionally, commons-logging was previously handled through the rest
client, but now xpack depends directly on this, thus excludes must be
added for the pesky missing classes there.

This commit also cleans up unnecessary parts of plugin/build.gradle no
longer necessary.

Original commit: elastic/x-pack-elasticsearch@70e936bdc3
2018-01-22 22:58:34 -08:00