Commit Graph

106 Commits

Author SHA1 Message Date
jaymode a547049303 Security: switch to IllegalArgumentException if request cannot be serialized
The commit switches the CreateTokenRequest and InvalidateTokenRequest to throw an
IllegalArgumentException when the version the request is being serialized to does not support the
request. This fixes test failures due to the use of the AssertingTransportInterceptor testing
serialization with versions prior to 6.2. The IAE is an indication to the
AssertingTranpsortInterceptor that the request does not support the version.

Original commit: elastic/x-pack-elasticsearch@c73abf1bc0
2018-01-24 12:19:09 -07:00
David Roberts ae1a90eba6 [ML] Move JobProvider and the Batched*Iterators out of core (elastic/x-pack-elasticsearch#3712)
By moving these into the ML module:

1. The classes are in the same module as their unit tests
2. We can extend the JobProvider in the future with functionality
   that is not in core

Original commit: elastic/x-pack-elasticsearch@610a89a3d9
2018-01-24 18:56:30 +00:00
Jay Modi 0baa45d9b3 [SAML] Find all tokens for a realm, not just the first 10 (elastic/x-pack-elasticsearch#3689)
This commit changes the token service to use a scroll based approach when finding all tokens by
the realm. Without this, we may only find a few tokens and leave some active that need to be
invalidated.

relates elastic/x-pack-elasticsearch#3688

Original commit: elastic/x-pack-elasticsearch@20e97b6aae
2018-01-24 11:07:51 -07:00
Costin Leau 3f4c789993 Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@348f0468b0
2018-01-24 19:19:51 +02:00
David Kyle 29ece3c79f [ML] Refactor update calendar params (elastic/x-pack-elasticsearch#3714)
Original commit: elastic/x-pack-elasticsearch@a7ef33b58a
2018-01-24 15:16:21 +00:00
Jason Tedor 3932635f98 Simplify security manager permissions
The security manager permissions were copied wholesale from pre-split
X-Pack. However, this grants unnecessary permissions to the child
plugins. This commit is a simple attempt at removing permissions that
are not needed in the child plugins.

Relates elastic/x-pack-elasticsearch#3651

Original commit: elastic/x-pack-elasticsearch@8325ed83d7
2018-01-24 08:57:54 -05:00
Tanguy Leroux ba7b84d4d0 Fix XPackExtension javadoc (elastic/x-pack-elasticsearch#3711)
Original commit: elastic/x-pack-elasticsearch@bbe950ce40
2018-01-24 14:24:39 +01:00
Igor Motov de15f7123b Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@bbabfb1328
2018-01-23 15:41:59 -05:00
Jason Tedor c0790d6a49 Move x-pack-core to core package (elastic/x-pack-elasticsearch#3678)
This commit moves the source file in x-pack-core to a org.elasticsearch.xpack.core package. This is to prevent issues where we have compile-time success reaching through packages that will cross module boundaries at runtime (due to being in different classloaders). By moving these to a separate package, we have compile-time safety. Follow-ups can consider build time checking that only this package is defined in x-pack-core, or sealing x-pack-core until modules arrive for us.

Original commit: elastic/x-pack-elasticsearch@232e156e0e
2018-01-23 12:43:58 -06:00
Igor Motov 2be9cabbe9 Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@d6451b0f6b
2018-01-23 12:53:13 -05:00
Alexander Reelsen d065b087ee Revert "Cleanup: Remove HaltedClock (elastic/x-pack-elasticsearch#3664)"
This reverts commit elastic/x-pack-elasticsearch@f91c401a60 due to
failing tests, like

./gradlew :x-pack-elasticsearch:plugin:watcher:test -Dtests.seed=AE30350FCE96D26D -Dtests.class=org.elasticsearch.xpack.watcher.watch.WatchTests -Dtests.method="testParserSelfGenerated" -Dtests.security.manager=true -Dtests.locale=ja-JP -Dtests.timezone=EET

Original commit: elastic/x-pack-elasticsearch@e45d79d643
2018-01-23 18:08:54 +01:00
David Roberts 697a08e742 [ML] Refactor so ML doesn't require PersistentTasksService at startup (elastic/x-pack-elasticsearch#3682)
At present the PersistentTasksService is created inside the ML plugin.
This is undesirable, as other plugins will use persistent tasks in the
near future.

This change refactors the startup code so that the PersistentTasksService
no longer needs to be passed to any constructors for ML components.

A future change will still be required to actually move the initialization
of the PersistentTasksClusterService, PersistentTasksService and
PersistentTasksExecutorRegistry out of the ML plugin, but following this
change it should be fairly simple.

Original commit: elastic/x-pack-elasticsearch@3c2a8e020e
2018-01-23 16:55:08 +00:00
Alexander Reelsen e3da8fa4ae Cleanup: Remove HaltedClock (elastic/x-pack-elasticsearch#3664)
The HaltedClock was a leftover from moving over from our own Clock
implementation to a java.time one. java.time already has a fixed clock,
this one is not needed.

Original commit: elastic/x-pack-elasticsearch@f91c401a60
2018-01-23 14:13:22 +01:00
Simon Willnauer c70c1c7e98 Expose XPackExtensions via SPI (elastic/x-pack-elasticsearch#3530)
This change adds SPI loading for XPackExtensions that allows to extend
XPack via an ordinary plugin. This can co-exist with the existin
extension mechanism for the time being.

Original commit: elastic/x-pack-elasticsearch@bf02b56dee
2018-01-23 13:05:39 +01:00
Ryan Ernst 9d87b63ca4 Build: Fix third party audit task for xpack core (elastic/x-pack-elasticsearch#3656)
This commit re-enables thirdPartyAudit for x-pack core. Previously, when
xpack was a single plugin, it transitively picked up httpcore-nio
through the elasticsearch rest client. Now that xpack core does not
depend on the rest client, httpcore-nio must be added as a dependency.
Additionally, commons-logging was previously handled through the rest
client, but now xpack depends directly on this, thus excludes must be
added for the pesky missing classes there.

This commit also cleans up unnecessary parts of plugin/build.gradle no
longer necessary.

Original commit: elastic/x-pack-elasticsearch@70e936bdc3
2018-01-22 22:58:34 -08:00
Igor Motov 6330209e7d Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@95cbd93803
2018-01-22 16:03:56 -05:00
Jason Tedor 520b372549 Remove watcher as a security test dependency
This commit removes Watcher as a test dependency for Security.

Relates elastic/x-pack-elasticsearch#3655

Original commit: elastic/x-pack-elasticsearch@8d770ffb9d
2018-01-22 10:16:03 -05:00
Luca Cavanna d9eb4c049f Trim down usages of `ShardOperationFailedException` interface (elastic/x-pack-elasticsearch#3662)
In many cases we use the `ShardOperationFailedException` interface to abstract an exception that can only be of one type, namely `DefaultShardOperationException`. There is no need to use the interface in such cases, the concrete type should be used instead. That has the additional advantage of simplifying parsing such exceptions back from rest responses for the high-level REST client.

Original commit: elastic/x-pack-elasticsearch@b2259afcbf
2018-01-22 15:51:55 +01:00
Dimitris Athanasiou cc432aee34 [ML] Check calendar exists before removing job (elastic/x-pack-elasticsearch#3661)
Also, removes check for whether a job-to-remove exists
and replaces it with a check of whether a job-to-remove
is already present in the calendar. This allows to
remove a job that may no longer exists and it improves
feedback for the case that an existing job is removed from
a calendar that doesn't contain it.

relates elastic/x-pack-elasticsearch#3620

Original commit: elastic/x-pack-elasticsearch@3ea39be1b6
2018-01-22 13:49:38 +00:00
Ryan Ernst ac44ee8257 Build: Move test files from plugin/src to appropriate other projects (elastic/x-pack-elasticsearch#3653)
This commit moves mostly security tests that were left in plugin/src. It
also moves the CompositeTestingXPackPlugin into tribe license tests qa
project, which was the only remaining use. This class needs to be
removed, as it has unavoidable problems with dependency conflicts (ie
guava) between security and watcher, which it pulls in both into the
test classpath.

Original commit: elastic/x-pack-elasticsearch@756209e010
2018-01-21 14:08:47 -05:00
Jason Tedor 4d4f979d61 Remove debugging println from API JAR task
This commit removes a leftover println that was added while debugging
the execution of the API JAR task.

Original commit: elastic/x-pack-elasticsearch@37c2e8fe5b
2018-01-21 13:20:03 -05:00
Jason Tedor 04b48324aa Drop native controller from descriptors (except ML)
These were copied wholesale from the pre-split X-Pack
descriptor. However, only ML has a native controller. This removes the
plugin installation asking multiple times to approve the existence of a
native controlled for every bundled plugin.

Relates elastic/x-pack-elasticsearch#3650

Original commit: elastic/x-pack-elasticsearch@4fca606243
2018-01-21 11:00:51 -05:00
Jason Tedor a582acddf8 Add API JAR task to x-pack-core
This commit adds back the task for assembling the API JAR from the
x-pack-core JAR.

Original commit: elastic/x-pack-elasticsearch@10385ecf17
2018-01-21 10:00:47 -05:00
Tim Vernum 47213f5675 [Security] Add SAML authentication support (elastic/x-pack-elasticsearch#3646)
Introduces:

- SAML Realm
- REST & Transport actions to support SAML single signon / signout
- Tests for above
- More XML than you ever wanted to see.

Original commit: elastic/x-pack-elasticsearch@b0fe7bb652
2018-01-21 08:43:00 +10:00
Jason Tedor 50864eabce Introduce plugin-specific env scripts
With plugins of a meta-plugin now loaded in separate classloaders, we
should not be loading all classes in all plugins of a meta-plugin when
executing scripts. This is particularly problematic in the case of
security extensions where the install plugin extension command would be
running with the classpath of all plugins. However, if there is JAR hell
in this classpath, installation would fail. This is not realistic though
since the plugins are run in separate classloaders. To fix this, for the
scripts of a plugin, we only set the classpath to include the JARs for
that plugin and the JARs of core. This leads us to the introduction of
plugin-specific env scripts.

Relates elastic/x-pack-elasticsearch#3649

Original commit: elastic/x-pack-elasticsearch@543df37eed
2018-01-20 15:56:00 -05:00
Jason Tedor 60e577a9c4 Fix test base class that disables ML autodetect
This is no longer needed for general X-Pack since the tests will not be
depending on ML. We move this class to an ML specific directory and
remove the dependency from other tests.

Original commit: elastic/x-pack-elasticsearch@9b287f7460
2018-01-20 01:27:33 -05:00
Michael Basnight f3ec4a5208 Split up xpack plugins into their own modules (elastic/x-pack-elasticsearch#3643)
Thanks to some great work by a bunch of amazing people, the chuck norris xpack split is a go!

Original commit: elastic/x-pack-elasticsearch@dad98e28f4
2018-01-19 23:30:17 -06:00
Dimitrios Athanasiou 9e834cbf8a [ML] Fix javadoc in MlStrings#hasValidLengthForId
Original commit: elastic/x-pack-elasticsearch@ef5ecad5e2
2018-01-19 11:45:41 +00:00
Dimitris Athanasiou 21f692c02b [ML] Further validate calendar_id and add calendar description (elastic/x-pack-elasticsearch#3624)
relates elastic/x-pack-elasticsearch#3595

Original commit: elastic/x-pack-elasticsearch@fade977361
2018-01-19 10:44:39 +00:00
Lee Hinman 5818986775 Merge remote-tracking branch 'origin/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@8475419a00
2018-01-18 16:51:56 -07:00
Dimitris Athanasiou eefd8e7940 [ML] Allow categorical conditions to use all analysis fields (elastic/x-pack-elasticsearch#3615)
Categorical conditions should be allowed to use any of the analysis
fields as a field_name.

Original commit: elastic/x-pack-elasticsearch@a0d25e7445
2018-01-18 15:48:56 +00:00
Jay Modi 60d4b7e53e Add the ability to refresh tokens obtained via the API (elastic/x-pack-elasticsearch#3468)
This commit adds the ability to refresh tokens that have been obtained by the API using a refresh
token. Refresh tokens are one time use tokens that are valid for 24 hours. The tokens may be used
to get a new access and refresh token if the refresh token has not been invalidated or
already refreshed.

relates elastic/x-pack-elasticsearch#2595

Original commit: elastic/x-pack-elasticsearch@23435eb815
2018-01-17 12:18:44 -07:00
Dimitris Athanasiou f91631664a [ML] Rename Condition member valueFilter -> value (elastic/x-pack-elasticsearch#3599)
Original commit: elastic/x-pack-elasticsearch@5ef8415ca5
2018-01-17 18:39:47 +00:00
Costin Leau 448511e093 Merge remote-tracking branch 'remotes/upstream/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@2c78f17486
2018-01-16 17:11:37 +02:00
Dimitrios Athanasiou 0b8723015f [ML] Capitalize first letter in error message
Original commit: elastic/x-pack-elasticsearch@7062851357
2018-01-16 11:50:39 +00:00
Dimitris Athanasiou 2350b85507 [ML] Improve error on get calendar when ml-meta index is missing (elastic/x-pack-elasticsearch#3576)
Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@bc60a0cb72
2018-01-16 11:00:07 +00:00
Dimitris Athanasiou 6c6dcafd0e [ML] Improve error message when post-calendar-events-request is empty (elastic/x-pack-elasticsearch#3572)
Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@3d25ed4723
2018-01-15 18:13:50 +00:00
Dimitris Athanasiou aff9a4a2ba [ML] Autodetect should receive events from the earliest valid timestamp (elastic/x-pack-elasticsearch#3570)
When events are searched to be passed to the autodetect process, they
are currently calculated based on the latest record timestamp, when
a job opens, and `now` when the process is updated.

This commit changes both to be consistent and based on the earliest
valid timestamp for the job. The earliest valid timestamp is the
latest record timestamp minus the job latency.

Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@7f882ea053
2018-01-15 18:07:48 +00:00
Tim Brooks 5eef187b19 Modify x-pack to support elasticsearch-core jar (elastic/x-pack-elasticsearch#3544)
This is related to elastic/elasticsearch#27933. It modifies x-pack to be
compatible with the new elasticsearch-core jar.

Original commit: elastic/x-pack-elasticsearch@42227b6034
2018-01-15 09:59:03 -07:00
Igor Motov 70cea58262 Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@9ac41d008e
2018-01-12 15:35:56 -05:00
Igor Motov aba3f15d81 Add adding ability to associate an ID with tasks (elastic/x-pack-elasticsearch#3500)
X-Pack portion of elastic/elasticsearch#23250

Original commit: elastic/x-pack-elasticsearch@3c9a5d2d08
2018-01-12 15:34:28 -05:00
Dimitris Athanasiou 73f8559fca [ML] Update process when filters or calendars are updated (elastic/x-pack-elasticsearch#3385)
Relates elastic/x-pack-elasticsearch#3325

Original commit: elastic/x-pack-elasticsearch@9da4973cda
2018-01-12 17:48:07 +00:00
David Kyle 86e9f63b19 [ML] Expect an array of events in request (elastic/x-pack-elasticsearch#3537)
* Expect an array of events in request

Original commit: elastic/x-pack-elasticsearch@f60bc0b544
2018-01-11 22:16:09 +00:00
Igor Motov 915ed35e27 Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@02fddccd84
2018-01-11 13:41:40 -05:00
David Roberts c647d7f38b [ML] Adjust BWC for categorization analyzer config after backporting elastic/x-pack-elasticsearch#3372
Original commit: elastic/x-pack-elasticsearch@27fcd5bdcb
2018-01-11 17:01:37 +00:00
David Roberts a386b5727e [ML] Increase tokenization flexibility for categorization (elastic/x-pack-elasticsearch#3372)
By moving tokenization for categorization to Java we give users access to considerably more options for tokenizing their log messages prior to using ML to categorize them. Now all Elasticsearch analyzer functionality is available, which opens up the possibility to sensibly categorize non-English log messages.

Relates elastic/machine-learning-cpp#491

Original commit: elastic/x-pack-elasticsearch@5d61b67614
2018-01-11 15:40:13 +00:00
David Kyle 4c973c16c2 [ML] Rename event filter params (elastic/x-pack-elasticsearch#3525)
Original commit: elastic/x-pack-elasticsearch@3089193d45
2018-01-10 14:40:32 +00:00
David Kyle 66b187fe17 [ML] Account for job groups membership when getting events (elastic/x-pack-elasticsearch#3473)
* Account for job groups membership when getting events

Original commit: elastic/x-pack-elasticsearch@a4185cc460
2018-01-10 11:14:09 +00:00
Dimitris Athanasiou aa25704170 [ML] Add scheduled_events field to bucket (elastic/x-pack-elasticsearch#3513)
Adds `scheduled_events` to the bucket object so
it can read the corresponding field written from the
autodetect process.

Relates elastic/x-pack-elasticsearch#3016

Original commit: elastic/x-pack-elasticsearch@3f35f867f0
2018-01-10 10:57:09 +00:00
David Kyle f73a7803ce [ML] Delete calendar events endpoint (elastic/x-pack-elasticsearch#3388)
* Delete calendar events endpoint

Original commit: elastic/x-pack-elasticsearch@70aebfae2c
2018-01-09 11:55:36 +00:00
David Kyle 2eb3f02e40 [ML] Rename SpecialEvents -> ScheduledEvents (elastic/x-pack-elasticsearch#3485)
* Rename SpecialEvents -> ScheduledEvents


Original commit: elastic/x-pack-elasticsearch@4bfc52c435
2018-01-05 16:35:42 +00:00
Igor Motov b769007798 Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@14750a59a8
2018-01-05 10:37:46 -05:00
David Kyle 46c1272ed3 [ML] Fix broken backwards compatibility of field names (elastic/x-pack-elasticsearch#3463)
Original commit: elastic/x-pack-elasticsearch@613943dfe7
2018-01-04 10:05:16 +00:00
Igor Motov 292506526e Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@79e7b1b953
2017-12-27 21:46:00 -05:00
Michael Basnight 220aa734ee Migrate actions to plugin core (elastic/x-pack-elasticsearch#3424)
This commit hacks up the xpack repo in such a way that it moves all
actions that the transport client uses to plugin core. It also moves
any classes that those actions use to plugin core, with a few
exceptions. I tried to split up any classes that pulled in server side
logic into the client, but that was not always 100% possible. Consider
this commit a guide, and since I do not know the codebase for each
plugin, consider it a best guess for what should be moved. A few other
things were merged in below.

This commit extracts the parts of Condition and AlwaysCondition that are
needed by classes that will be moved into the client package. The only
odd thing here is that since there are two parent classes (Condition and
AlwaysCondition) it was not possible to make AlwaysConditionInteral
extend from AlwaysCondition and still parse a ConditionInternal object,
which is what the ConditionFactory expects.

This commit removes the use of internal users in the User class, and
instead moves them to Authentication, where they are used.

[insert obligatory chuck norris karate image here]

ref elastic/x-pack-elasticsearch#2925

Original commit: elastic/x-pack-elasticsearch@42d0b72209
2017-12-27 11:31:16 -05:00
Lee Hinman 86a04acb01 Rename folder x-pack-core -> core (elastic/x-pack-elasticsearch#3305)
* Rename folder x-pack-core -> core

The jar remains 'x-pack-core-*.jar'

* Put group in top-level build.gradle instead of plugin/core/build.gradle

Original commit: elastic/x-pack-elasticsearch@b23452fa55
2017-12-12 13:23:29 -07:00