Commit Graph

1302 Commits

Author SHA1 Message Date
David Kyle 938cf239c9 [M] Clean noisy error message in Yaml tests (elastic/x-pack-elasticsearch#3923)
Original commit: elastic/x-pack-elasticsearch@5f0d4040dc
2018-02-13 17:05:27 +00:00
David Kyle 8e73085047 [ML] Enable adding multiple jobs to a calendar (elastic/x-pack-elasticsearch#3786)
Original commit: elastic/x-pack-elasticsearch@56a70a4580
2018-02-08 11:44:16 +00:00
David Roberts f4b32bef8c [ML] Reject headers supplied directly in the put datafeed body (elastic/x-pack-elasticsearch#3853)
Since elastic/x-pack-elasticsearch#3254 security headers have been stored in datafeed cluster state
to allow the datafeed to run searches using the credentials of the user
who created/updated it.  As a result the parser was changed to read the
"headers" field so that cluster state could be reloaded.  However, this
meant that datafeed configs could be submitted with a "headers" field.
No security loophole arose from this, as subsequent code overwrites the
contents of any supplied headers.  But it could be confusing that an
erroneously supplied field did not cause a parse failure as it usually
would.

This change makes the config parser for datafeeds reject a "headers"
field.  Now only the metadata parser used for reloading cluster state
will read a "headers" field.

Original commit: elastic/x-pack-elasticsearch@afa503275f
2018-02-08 10:43:33 +00:00
Lee Hinman 1b36133988 Merge remote-tracking branch 'origin/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@4176773659
2018-01-31 10:49:25 -07:00
Dimitris Athanasiou ed11dad855 [ML] Expand detector function shortcuts (elastic/x-pack-elasticsearch#3789)
relates elastic/x-pack-elasticsearch#3162


Original commit: elastic/x-pack-elasticsearch@a3512c6693
2018-01-31 13:11:32 +00:00
Lee Hinman a789944b8d Merge remote-tracking branch 'origin/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@f96f80f0d0
2018-01-29 15:42:01 -07:00
Costin Leau 07658cc04f SQL: replace JDBC transport meta columns with SQL call (elastic/x-pack-elasticsearch#3740)
Replace meta column endpoint with SYS COLUMNS command

Original commit: elastic/x-pack-elasticsearch@819874bc5b
2018-01-29 19:18:24 +02:00
Costin Leau e929d16d76 SQL: Replace ListTable transport with SQL command (elastic/x-pack-elasticsearch#3675)
Introduce system commands as alternative to meta HTTP endpoints
Pass in cluster name
Use 'BASE TABLE' instead of 'INDEX' when describing a table to stick
with the SQL terminology

Original commit: elastic/x-pack-elasticsearch@600312b8f7
2018-01-27 11:54:19 +02:00
Dimitris Athanasiou 867b4fe9fc [ML] Ensure BWC between old and new rules syntax (elastic/x-pack-elasticsearch#3738)
relates elastic/x-pack-elasticsearch#3325

Original commit: elastic/x-pack-elasticsearch@74cccbccf0
2018-01-26 15:52:01 +00:00
Igor Motov de15f7123b Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@bbabfb1328
2018-01-23 15:41:59 -05:00
Jason Tedor c0790d6a49 Move x-pack-core to core package (elastic/x-pack-elasticsearch#3678)
This commit moves the source file in x-pack-core to a org.elasticsearch.xpack.core package. This is to prevent issues where we have compile-time success reaching through packages that will cross module boundaries at runtime (due to being in different classloaders). By moving these to a separate package, we have compile-time safety. Follow-ups can consider build time checking that only this package is defined in x-pack-core, or sealing x-pack-core until modules arrive for us.

Original commit: elastic/x-pack-elasticsearch@232e156e0e
2018-01-23 12:43:58 -06:00
Igor Motov 2be9cabbe9 Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@d6451b0f6b
2018-01-23 12:53:13 -05:00
David Kyle 97e018f1bd [ML] Return the updated calendar after removing a job (elastic/x-pack-elasticsearch#3690)
Original commit: elastic/x-pack-elasticsearch@1bf1f3228b
2018-01-23 17:40:41 +00:00
David Kyle 8f393c3066 [ML] Improve error message when creating calendars (elastic/x-pack-elasticsearch#3668)
Original commit: elastic/x-pack-elasticsearch@996b0e2f65
2018-01-23 09:34:07 +00:00
Igor Motov ae55f506ca SQL: Move IT tests sql plugin project
Original commit: elastic/x-pack-elasticsearch@c7b59f05a5
2018-01-22 19:11:32 -05:00
Igor Motov 6330209e7d Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@95cbd93803
2018-01-22 16:03:56 -05:00
Dimitris Athanasiou cc432aee34 [ML] Check calendar exists before removing job (elastic/x-pack-elasticsearch#3661)
Also, removes check for whether a job-to-remove exists
and replaces it with a check of whether a job-to-remove
is already present in the calendar. This allows to
remove a job that may no longer exists and it improves
feedback for the case that an existing job is removed from
a calendar that doesn't contain it.

relates elastic/x-pack-elasticsearch#3620

Original commit: elastic/x-pack-elasticsearch@3ea39be1b6
2018-01-22 13:49:38 +00:00
Dimitris Athanasiou f3282b559f [ML] Check calendar exists before deleting event (elastic/x-pack-elasticsearch#3659)
Solves the event part of elastic/x-pack-elasticsearch#3620

Original commit: elastic/x-pack-elasticsearch@c79ca85c6e
2018-01-22 13:02:42 +00:00
Ryan Ernst ac44ee8257 Build: Move test files from plugin/src to appropriate other projects (elastic/x-pack-elasticsearch#3653)
This commit moves mostly security tests that were left in plugin/src. It
also moves the CompositeTestingXPackPlugin into tribe license tests qa
project, which was the only remaining use. This class needs to be
removed, as it has unavoidable problems with dependency conflicts (ie
guava) between security and watcher, which it pulls in both into the
test classpath.

Original commit: elastic/x-pack-elasticsearch@756209e010
2018-01-21 14:08:47 -05:00
Tim Vernum 47213f5675 [Security] Add SAML authentication support (elastic/x-pack-elasticsearch#3646)
Introduces:

- SAML Realm
- REST & Transport actions to support SAML single signon / signout
- Tests for above
- More XML than you ever wanted to see.

Original commit: elastic/x-pack-elasticsearch@b0fe7bb652
2018-01-21 08:43:00 +10:00
Michael Basnight f3ec4a5208 Split up xpack plugins into their own modules (elastic/x-pack-elasticsearch#3643)
Thanks to some great work by a bunch of amazing people, the chuck norris xpack split is a go!

Original commit: elastic/x-pack-elasticsearch@dad98e28f4
2018-01-19 23:30:17 -06:00
Jason Tedor 0602d38bd5 Split transport implementations into client/server
This commit splits the transport implementations into components that
can be used client-side (in the transport client) and server-side (in
the server). This enables removing security as a dependency for the
transport client.

Relates elastic/x-pack-elasticsearch#3635

Original commit: elastic/x-pack-elasticsearch@e480eb7eb2
2018-01-19 12:09:34 -05:00
Dimitris Athanasiou 21f692c02b [ML] Further validate calendar_id and add calendar description (elastic/x-pack-elasticsearch#3624)
relates elastic/x-pack-elasticsearch#3595

Original commit: elastic/x-pack-elasticsearch@fade977361
2018-01-19 10:44:39 +00:00
Lee Hinman 5818986775 Merge remote-tracking branch 'origin/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@8475419a00
2018-01-18 16:51:56 -07:00
Tim Brooks 685b75da3a Support changes in nio channel contexts (elastic/x-pack-elasticsearch#3609)
This is related to elastic/elasticsearch#elastic/x-pack-elasticsearch#28275. It modifies x-pack to
support the changes in channel contexts. Additionally, it simplifies
the SSLChannelContext by relying on some common work between it and
BytesChannelContext.

Original commit: elastic/x-pack-elasticsearch@8a8fcce050
2018-01-18 13:06:42 -07:00
jaymode e775e84a7e Test: add additional logging to SslMultiPortTests
This commit adds some additional logging to the SslMultiPortTests to help with debugging an
intermittent failure.

Relates elastic/x-pack-elasticsearch#1809

Original commit: elastic/x-pack-elasticsearch@ae9c967249
2018-01-18 12:46:01 -07:00
jaymode 40d315b244 Test: only use a single shard in RemoteIndexAuditTrailStartingTests
Relates elastic/x-pack-elasticsearch#435

Original commit: elastic/x-pack-elasticsearch@05db61297b
2018-01-18 12:07:30 -07:00
Tim Brooks fb12a0e383 Support TLS/SSL renegotiation (elastic/x-pack-elasticsearch#3600)
This commit is related to elastic/x-pack-elasticsearch#3246. It adds support for receiving TLS/SSL
renegotiation requests for peers.

Original commit: elastic/x-pack-elasticsearch@c22c16b3bc
2018-01-18 10:59:44 -07:00
Dimitris Athanasiou eefd8e7940 [ML] Allow categorical conditions to use all analysis fields (elastic/x-pack-elasticsearch#3615)
Categorical conditions should be allowed to use any of the analysis
fields as a field_name.

Original commit: elastic/x-pack-elasticsearch@a0d25e7445
2018-01-18 15:48:56 +00:00
Jay Modi 60d4b7e53e Add the ability to refresh tokens obtained via the API (elastic/x-pack-elasticsearch#3468)
This commit adds the ability to refresh tokens that have been obtained by the API using a refresh
token. Refresh tokens are one time use tokens that are valid for 24 hours. The tokens may be used
to get a new access and refresh token if the refresh token has not been invalidated or
already refreshed.

relates elastic/x-pack-elasticsearch#2595

Original commit: elastic/x-pack-elasticsearch@23435eb815
2018-01-17 12:18:44 -07:00
Igor Motov c3b82e5ee1 SQL: Remove test-utils project (elastic/x-pack-elasticsearch#3583)
Makes Sql Translate Action consistent with other SQL Actions and removes test-utils project

Follow up for elastic/x-pack-elasticsearch#3543

Original commit: elastic/x-pack-elasticsearch@8ff2148d67
2018-01-17 12:18:25 -05:00
Tim Brooks dda3a8dee0 Add TLS/SSL enabled SecurityNioTransport (elastic/x-pack-elasticsearch#3519)
This is related to elastic/x-pack-elasticsearch#3246. This commit adds a SSL/TLS layer to the nio
work implemented in the SSLChannelContext and SSLDriver classes.
This work is used to build up a SecurityNioTransport implementation.
This transport does yet offer feature parity with our normal security
transport. It mainly offers SSL/TLS security.

Original commit: elastic/x-pack-elasticsearch@d0e0484418
2018-01-17 09:44:31 -07:00
Alexander Reelsen ef2d2764a5 Watcher: Improve cluster state listener behaviour (elastic/x-pack-elasticsearch#3538)
The cluster state listener used by watch now have two additional checks.
First, when no master node exists in the cluster state, watcher will
stop and the indexing listener will not try to trigger any new watch.
Second, when there is a global cluster write level block, it would not
be possible to update the watches index or write into the watcher
history, so the listener can bail at that case as well.

In addition this also changes the log level from debug to info when
watcher is stopped. It turned out that there are zero insights when or
if watcher is stopped when normal logging is activated. This makes it
super hard for support to know when watcher is stopped or started at all
due to shards being moved around.

Original commit: elastic/x-pack-elasticsearch@5e9ce24380
2018-01-17 14:18:17 +01:00
Tim Vernum b0552e1c6e [Security] Handle cache expiry in token service (elastic/x-pack-elasticsearch#3565)
* [Security] Handle cache expiry in token service

The keyCache on TokenService.KeyAndCache has a 60 minute expiry.
If the token service was idle for more than 60 minutes, the current
key would be expired and it would then fail to generate user tokens.

Original commit: elastic/x-pack-elasticsearch@fd98130a27
2018-01-17 13:04:59 +10:00
Yogesh Gaikwad 29663c1f38 Fix for Issue elastic/x-pack-elasticsearch#3403 - Predictable ordering of security realms (elastic/x-pack-elasticsearch#3533)
* Security Realms: Predictable ordering for realms

To have predictable ordering of realms, by having secondary
sorting on realm name resulting in stable and consistent documentation.
Documentation update describing how ordering of realms is determined.
Testing done by adding unit test for the change, ran gradle clean check locally.

relates elastic/x-pack-elasticsearch#3403

Original commit: elastic/x-pack-elasticsearch@98c42a8c51
2018-01-17 10:29:00 +11:00
Igor Motov 12d4f10faa SQL: Switch JDBC to REST protocol (elastic/x-pack-elasticsearch#3543)
Replaces binary serialization protocol in JDBC driver with the common REST protocol.

relates elastic/x-pack-elasticsearch#3419

Original commit: elastic/x-pack-elasticsearch@532c106658
2018-01-16 13:26:06 -05:00
Costin Leau 448511e093 Merge remote-tracking branch 'remotes/upstream/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@2c78f17486
2018-01-16 17:11:37 +02:00
Dimitris Athanasiou 368c4fff56 [ML] Expand job groups on scheduled events update (elastic/x-pack-elasticsearch#3577)
Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@73334d8e01
2018-01-16 11:53:37 +00:00
Dimitrios Athanasiou 0b8723015f [ML] Capitalize first letter in error message
Original commit: elastic/x-pack-elasticsearch@7062851357
2018-01-16 11:50:39 +00:00
Dimitris Athanasiou 2350b85507 [ML] Improve error on get calendar when ml-meta index is missing (elastic/x-pack-elasticsearch#3576)
Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@bc60a0cb72
2018-01-16 11:00:07 +00:00
Alexander Reelsen ac2884f034 Watcher: Fix NPE in watcher index template registry (elastic/x-pack-elasticsearch#3571)
The current code throws an NPE, when there is no master node available
in the cluster state. This commit adds an additional check for an
existing master node.

relates elastic/x-pack-elasticsearch#3569

Original commit: elastic/x-pack-elasticsearch@e0024b5170
2018-01-16 10:11:12 +01:00
Dimitris Athanasiou 6c6dcafd0e [ML] Improve error message when post-calendar-events-request is empty (elastic/x-pack-elasticsearch#3572)
Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@3d25ed4723
2018-01-15 18:13:50 +00:00
Dimitris Athanasiou aff9a4a2ba [ML] Autodetect should receive events from the earliest valid timestamp (elastic/x-pack-elasticsearch#3570)
When events are searched to be passed to the autodetect process, they
are currently calculated based on the latest record timestamp, when
a job opens, and `now` when the process is updated.

This commit changes both to be consistent and based on the earliest
valid timestamp for the job. The earliest valid timestamp is the
latest record timestamp minus the job latency.

Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@7f882ea053
2018-01-15 18:07:48 +00:00
Tim Brooks 5eef187b19 Modify x-pack to support elasticsearch-core jar (elastic/x-pack-elasticsearch#3544)
This is related to elastic/elasticsearch#27933. It modifies x-pack to be
compatible with the new elasticsearch-core jar.

Original commit: elastic/x-pack-elasticsearch@42227b6034
2018-01-15 09:59:03 -07:00
Albert Zaharovits 93b8ef49e9 Fix CI failure elastic/x-pack-elasticsearch#3568
Bad test design & excessive randomness

Original commit: elastic/x-pack-elasticsearch@076d0b246a
2018-01-15 18:39:13 +02:00
Alexander Reelsen d4cddc12d0 Tests: Add @AwaitsFix annotation to flaky tests
These contribute to unstable builds, and should be reenabled on a
case by case base and worked on.

Original commit: elastic/x-pack-elasticsearch@a15338415a
2018-01-15 10:28:05 +01:00
Igor Motov 70cea58262 Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@9ac41d008e
2018-01-12 15:35:56 -05:00
Igor Motov aba3f15d81 Add adding ability to associate an ID with tasks (elastic/x-pack-elasticsearch#3500)
X-Pack portion of elastic/elasticsearch#23250

Original commit: elastic/x-pack-elasticsearch@3c9a5d2d08
2018-01-12 15:34:28 -05:00
Jay Modi cec90f452a IndexLifecycleManager checks for index existense and up to date mappings (elastic/x-pack-elasticsearch#3515)
This change makes the IndexLifecycleManager check for both index existence and up to date mappings
on the index prior to executing the provided runnable. Doing this provides a mechanism to make
non-breaking mapping updates to the security index in minor versions.

relates elastic/x-pack-elasticsearch#3462

Original commit: elastic/x-pack-elasticsearch@80f05d83b4
2018-01-12 12:45:52 -07:00
Dimitris Athanasiou 73f8559fca [ML] Update process when filters or calendars are updated (elastic/x-pack-elasticsearch#3385)
Relates elastic/x-pack-elasticsearch#3325

Original commit: elastic/x-pack-elasticsearch@9da4973cda
2018-01-12 17:48:07 +00:00