Commit Graph

7700 Commits

Author SHA1 Message Date
Nik Everett b95745956e Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@5f6bbb588a
2018-02-07 09:38:13 -05:00
Tim Vernum 7bb2cba14f [Security] Reset IndexAuditTrail to INITIALISED before start (elastic/x-pack-elasticsearch#3807)
Calling start() when already in the STARTING state doesn't do anything, so the component
gets stuck in STARTING state forever.

Also: wait on the required index name not just the cluster.

Also: added more logging to help diagnose such issues (either in RemoteIndexAuditTrailStartingTests or production)

Original commit: elastic/x-pack-elasticsearch@fb81214fe7
2018-02-07 12:30:23 +11:00
Nik Everett 10efd82694 SQL: Shift sql-cli into its proper home
The old home made sense before x-pack was split. The new home lines up
with where security keeps its scripts.

The jar file wasn't being included any more so this re-adds that.

Original commit: elastic/x-pack-elasticsearch@d3ec941397
2018-02-06 16:17:18 -05:00
jaymode 092fcd3675 Test: do not check cluster stats in audit qa project
This commit wraps the TestCluster in one that does not check stats to prevent tripping an
assertion for the accounting breaker as the index audit trail is running in the background and
will affect the value of the breakers.

See elastic/x-pack-elasticsearch#157

Original commit: elastic/x-pack-elasticsearch@381ebbd413
2018-02-06 13:35:09 -07:00
jaymode 4731a1b56b Test: disable monitoring for security spi qa project
This commit disables monitoring for the qa project that tests custom realms and role providers
as monitoring can cause failures due to an accounting breaker not being reset.

See elastic/x-pack-elasticsearch#157

Original commit: elastic/x-pack-elasticsearch@f882507e85
2018-02-06 13:09:09 -07:00
Nik Everett a2f1ed3f62 SQL: Fix javadoc in JdbcConfiguration
It wasn't building....

Original commit: elastic/x-pack-elasticsearch@059c6397a2
2018-02-06 14:41:06 -05:00
Nik Everett 5b46899cc3 Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@9a49ecbbd5
2018-02-06 14:40:55 -05:00
Jay Modi 18c04c714d [Security] handle null values when attempting to parse as DN when mapping roles (elastic/x-pack-elasticsearch#3791)
This commit adds special handling for null values when building a DistinguishedNamePredicate for
role mapping. Previously this would have resulted in an exception from the unboundid ldapsdk as
the DN is invalid.

relates elastic/x-pack-elasticsearch#3787

Original commit: elastic/x-pack-elasticsearch@9386dae03e
2018-02-06 10:19:49 -07:00
Nik Richers 20c1791dd8 Fix broken link for 6.2
Direct commit to speed up fix.

Original commit: elastic/x-pack-elasticsearch@f9187b81b8
2018-02-06 07:21:19 -08:00
Lee Hinman b2b95cb25d [TEST] Fix compilation for XContent parser
Original commit: elastic/x-pack-elasticsearch@bf03cc34fe
2018-02-05 16:56:36 -07:00
Lee Hinman 5a3004300c SQL: Add some Javadocs (elastic/x-pack-elasticsearch#3816)
* SQL: Add some Javadocs

* Reflow javadoc for fewer columns

Original commit: elastic/x-pack-elasticsearch@d23239b2e5
2018-02-05 09:03:06 -07:00
Tim Vernum 80b5ac9562 [DOC] SAML documentation (elastic/x-pack-elasticsearch#3657)
Includes:
- docs for new realm type "saml"
- docs for new settings for SAML realms
- a guide for setting up SAML accross ES + Kibana

Original commit: elastic/x-pack-elasticsearch@85f8f6d409
2018-02-05 12:22:54 +11:00
Nik Everett 876aebf7e0 SQL: Make extract work for any datetime function (elastic/x-pack-elasticsearch#3756)
This allows any datetime function to be present in `EXTRACT` which feels
more consistent. `EXTRACT(FOO FROM bar)` is now just sugar for
`FOO(bar)`. This is *much* simpler to explain in the documentation then
"these 10 fields are supported by extract and they are the same as this
subset of the datetime functions."

The implementation of this is a little simpler then the old way. Instead
of resolving the function in the parser we create an
`UnresolvedFunction` that looks *almost* just like what we'd create for
a single argument function and resolve the function in the `Analyzer`.
This feels like a net positive as it allows us to group `EXTRACT`
resolution failures with other function resolution failures.

This also creates `UnresolvedFunctionTests` and
`UnresolvedAttributeTests`. I had to create `UnresolvedFunctionTests`
because `UnreolvedFunction` now has three boolean parameters which is
incompatible with the generic `NodeSubclassTests`'s requirement that all
ctor parameters be unique. I created `UnresolvedAttributeTests` because
I didn't want `UnresolvedFunctionTests` to call `NodeSubclassTests` and
figured that we'd want `UnresolvedAttributeTest` eventually and now felt
like as good a time as any.

Added a 

Original commit: elastic/x-pack-elasticsearch@358aada308
2018-02-03 16:10:09 -05:00
Nik Everett 2a5eacfc0a SQL: Fix up docs around score
Explain what happens if you have multiple full text queries in your
`WHERE` clause.

Original commit: elastic/x-pack-elasticsearch@cbae17b465
2018-02-02 11:25:09 -05:00
Costin Leau 34fe0beb30 SQL: add comments on JDBC thread-safety (or rather lack of)
Original commit: elastic/x-pack-elasticsearch@6ea5adc6a8
2018-02-02 18:11:24 +02:00
Nik Everett 2c66b058f3 SQL: More comments in docs
Leave more comments in docs for things that we have to finish
documenting.

Original commit: elastic/x-pack-elasticsearch@2523617c3b
2018-02-02 11:10:34 -05:00
Nik Everett 8065b71bc6 SQL: Drop double quotes from permissions in docs
We don't need the double quotes. Also, we follow up with an example that
shows how to write them in yml.

Original commit: elastic/x-pack-elasticsearch@835deca6f9
2018-02-02 11:07:54 -05:00
Nik Everett 6808eabc7b SQL: Comment out TODOs in docs
That way we don't render them to users. We already are tracking them in
issue elastic/x-pack-elasticsearch#2898.

Original commit: elastic/x-pack-elasticsearch@de506faad0
2018-02-02 11:06:35 -05:00
Costin Leau bda97c6e59 SQL: More polish
Original commit: elastic/x-pack-elasticsearch@cedbad466b
2018-02-02 18:01:51 +02:00
Costin Leau b5dd4c649a SQL: Minor fixes
Original commit: elastic/x-pack-elasticsearch@14ea747e20
2018-02-02 17:50:33 +02:00
Costin Leau 817c4759c6 Remove unused classes
Original commit: elastic/x-pack-elasticsearch@f7aee26821
2018-02-02 17:14:54 +02:00
Costin Leau d51a7192af [DOCS] Polish
Original commit: elastic/x-pack-elasticsearch@cc82fb428e
2018-02-02 17:10:32 +02:00
lcawley 66414afb30 [DOCS] Updated doc info in readme
Original commit: elastic/x-pack-elasticsearch@89b4ec79ba
2018-02-01 18:37:01 -08:00
Nik Everett f66e01369a Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@6ba2651f46
2018-02-01 17:21:37 -05:00
Nik Everett c53e1f4b1c SQL: More tests and docs for permissions (elastic/x-pack-elasticsearch#3679)
Adds tests for what works and what doesn't when we're missing some of
SQL's required permissions.

Adds required permissions to the documentation of each SQL access
method.

relates elastic/x-pack-elasticsearch#3552

Original commit: elastic/x-pack-elasticsearch@971dabb3b4
2018-02-01 17:20:44 -05:00
Lisa Cawley 83910f717e [DOCS] Build docs from elasticsearch (elastic/x-pack-elasticsearch#3638)
Original commit: elastic/x-pack-elasticsearch@379e93a887
2018-02-01 13:35:42 -08:00
David Turner a1cb0911d3 Allow use of LocalDateTime#atZone() in DateTimeFormatterTimestampConverter (elastic/x-pack-elasticsearch#3811)
Adds a `@SuppressForbidden` annotation to permit use of a
trappy method from `java.time`, pending resolution of elastic/x-pack-elasticsearch#3810.

X-pack component of elasticsearch#28476.

Original commit: elastic/x-pack-elasticsearch@c5e1ae0970
2018-02-01 14:33:23 +00:00
Martijn van Groningen ee834026d1 Merge changes related to moving persistent tasks to elasticsearch repo into master branch.
Original commit: elastic/x-pack-elasticsearch@fb61e4853c
2018-02-01 15:31:57 +01:00
Dimitrios Athanasiou 05515b370a [ML][TEST] Increase timeout in MlDailyManagementServiceTests
relates elastic/x-pack-elasticsearch#2916

Original commit: elastic/x-pack-elasticsearch@2925dc45ea
2018-02-01 14:09:33 +00:00
Jason Tedor 2a7015a12c Bump Gradle to version 4.5
This commit bumps Gradle from version 4.3 to version 4.5.

Relates elastic/x-pack-elasticsearch#3805

Original commit: elastic/x-pack-elasticsearch@5e464a7d2b
2018-01-31 23:18:58 -05:00
Tim Vernum 415bb7f039 Clear Realm Caches on role mapping health change (elastic/x-pack-elasticsearch#3782)
If any of the follow take place on security index, then any cached role mappings
are potentially invalid and the associated realms need to clear any cached users.
- Index recovers from red
- Index is deleted
- Index becomes out-of-date / not-out-of-date

Original commit: elastic/x-pack-elasticsearch@1bcd86fcd4
2018-02-01 10:01:57 +11:00
Lee Hinman 1b36133988 Merge remote-tracking branch 'origin/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@4176773659
2018-01-31 10:49:25 -07:00
Tal Levy a627fec53e fix timeout usage in _bulk to `timeout` from `master_timeout` (elastic/x-pack-elasticsearch#3796)
The HTTP Exporter in Monitoring allowed users to set a timeout parameters
for the requests. When set, this was setting the `master_timeout` query parameter
in Bulk Requests. The problem is that Bulk Requests do not support this type of
timeout.

Original commit: elastic/x-pack-elasticsearch@9be194006e
2018-01-31 09:13:46 -08:00
Martijn van Groningen d2e7cee41d fixed compile error
Original commit: elastic/x-pack-elasticsearch@ff02a3244c
2018-01-31 15:38:14 +01:00
Martijn van Groningen 5ed8d81df8 Disabled integtest in core, because there are no *IT tests anymore.
Original commit: elastic/x-pack-elasticsearch@b4343d3ec4
2018-01-31 15:10:43 +01:00
Alexander Reelsen c20f3ba996 Watcher: Add support for scheme in proxy configuration (elastic/x-pack-elasticsearch#3614)
This adds support to allow different schemes in a proxy being used
compared to what the actual request requires. So if your proxy runs via
HTTP, but the endpoint you want to connect to uses HTTPS, this is now
possible to configure the proxy explicitely.

Also a small unit test for parsing this has been added.

relates elastic/x-pack-elasticsearch#3596

Original commit: elastic/x-pack-elasticsearch@176f7cdf0e
2018-01-31 14:12:25 +01:00
Dimitris Athanasiou ed11dad855 [ML] Expand detector function shortcuts (elastic/x-pack-elasticsearch#3789)
relates elastic/x-pack-elasticsearch#3162


Original commit: elastic/x-pack-elasticsearch@a3512c6693
2018-01-31 13:11:32 +00:00
Alexander Reelsen 71f68d3413 Tests: Remove watcher ensureWachterStarted/Stopped methods (elastic/x-pack-elasticsearch#3798)
Tests have been failing because of out of order cluster state processing
or because of stopping/starting was still in progress. Current tests do
not do further stop/start tries after the first leading to potential
being stuck in an unwanted state. This commit removes the methods
checking for the state being started or stopped in favor of using the
stopWatcher/startWatcher methods which now check if the desired state is
reached and otherwise issue another start/stop command.

Original commit: elastic/x-pack-elasticsearch@97b3232a6a
2018-01-31 12:15:19 +01:00
Martijn van Groningen b2e0cfb111 Cut ml over to `PersistentTaskPlugin` and drop its own initialization of the persistent tasks framework.
Original commit: elastic/x-pack-elasticsearch@484b53c2a8
2018-01-31 11:08:45 +01:00
Martijn van Groningen 6f101d71d5 Removed persistent tasks framework from xpack-elasticsearch repository, because it now part of the elasticsearch repository.
Original commit: elastic/x-pack-elasticsearch@7a90ffc4f6
2018-01-31 11:05:54 +01:00
Tim Vernum d0f6d273d0 [TEST] Add additional logging to test
Attempt to diagnose elastic/x-pack-elasticsearch#3292 if it occurs again.

Original commit: elastic/x-pack-elasticsearch@eb262540f0
2018-01-31 18:26:57 +11:00
Nik Everett 99081adfbf XContent: Adapt to new method on parser (elastic/x-pack-elasticsearch#3797)
Properly delegates `XContentParser#getDeprecationHandler`.

Original commit: elastic/x-pack-elasticsearch@45356d600a
2018-01-30 18:21:21 -05:00
Simon Willnauer 570411c2dc Remove all tribe related code, comments and documentation (elastic/x-pack-elasticsearch#3784)
Relates to elastic/elasticsearch#28443

Original commit: elastic/x-pack-elasticsearch@5c4e7fccc7
2018-01-30 20:40:46 +01:00
Nik Everett 116807ef54 SQL: Handle rename in core
Original commit: elastic/x-pack-elasticsearch@32cd41e6c8
2018-01-30 12:50:32 -05:00
Nik Everett 6631fe0376 Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@ec3a82494e
2018-01-30 12:50:27 -05:00
Dimitris Athanasiou 5b7c38da7f [ML] UpdateProcessNotifier should drain the update queue (elastic/x-pack-elasticsearch#3774)
The notifier is scheduled to run once per second. Currently,
it simply polls for the next update in the queue. However,
when there are multiple updates queued up, there is no
reason to wait for subsequent runs in order to execute the
rest of the updates.

This commit changes the notifier to drain the queue each time
it runs. It then serially executes the updates.

relates elastic/x-pack-elasticsearch#3769

Original commit: elastic/x-pack-elasticsearch@7a433c17f2
2018-01-30 16:35:02 +00:00
David Roberts ce77a3dd6d [TEST] Make test more tolerant to slow VMs (elastic/x-pack-elasticsearch#3783)
The test this change applies to has failed a couple of times on
Windows VMs.

Original commit: elastic/x-pack-elasticsearch@94e94d3ffd
2018-01-30 16:34:16 +00:00
Andy Bristol e45b738b4c [TEST] remove check for extensions executables (elastic/x-pack-elasticsearch#3778)
These executables have been removed because the old plugin
infrastructure has been replaced with SPI

Follow up from elastic/x-pack-elasticsearch#3734

Original commit: elastic/x-pack-elasticsearch@89dd1ab3b1
2018-01-30 07:42:44 -08:00
Jay Modi 71788671ad Test: fix skip check for EllipticCurveSSLTests (elastic/x-pack-elasticsearch#3771)
This change fixes the skip check for the EllipticCurveSSLTests. The skip check that is in the test
was added to proctect against failures on JVMs that do not support EC ciphers such as the packaged
openjdk in some linux distributions. The old skip check did not execute until the cluster was up
so the test would still fail with errors such as no cipher suites in common. This change moves the
check into a before class method that checks availability.

Original commit: elastic/x-pack-elasticsearch@5550ca4e1e
2018-01-30 08:21:15 -07:00
jaymode 0f6ca4716b Test: increase time to wait for the tribe node to start
See elastic/x-pack-elasticsearch#3754

Original commit: elastic/x-pack-elasticsearch@b1fb53ce8e
2018-01-30 08:13:19 -07:00