Commit Graph

1004 Commits

Author SHA1 Message Date
Gordon Brown 1f13c77b49 Merge branch 'master' into index-lifecycle 2018-08-23 11:52:59 -06:00
Zachary Tong 0da981a6a9
[TEST] Add some ACL yaml tests for Rollup (#33035)
These two tests compliment the existing unit tests which check Rollup's
ACL/security integration.

The first test creates to indices, puts a document in each one, and then
assigns a role to the test user that can only access one of the indices.
A rollup job is created with a pattern that would match both indices,
and we verify that only the allowed document was rolled up (e.g. verifying
that the unpermissioned index stays hidden).

The second test creates a single index with two documents tagged by
the keyword "public"/"private".  An attribute-based role is created
that only allows viewing "public" documents.  We then verify the rollup
job only rolled the "public" doc, and not the "private" one.
2018-08-23 11:43:48 -04:00
Colin Goodheart-Smithe fd88ab8c75
Fixes shrink action to remove single ndoe allocation (#33091)
This change fixes the shrink action so when the shrink is performed we
remove the single node allocation fromt eh shard allocation filtering
settings. Without this fix replicas cannot be allocated after we have
performed the shrink and we cannot make progress with the rest of the
shink aciton.

This change also fixes a bug in the explain API where the maste node
timeout was being set to null if it wasn't provided instead of using
its default value causing a NPE
2018-08-23 16:36:57 +01:00
Michael Basnight 644c0de5ec
Move non duplicated actions back into xpack core (#32952)
Most actions' request and response were moved from xpack core into
protocol. We have decided to instead duplicate the actions in the HLRC
instead of trying to reuse them. This commit moves the non duplicated
actions back into xpack core and severs the tie between xpack core and
protocol so no other actions can be moved and not duplicated.
2018-08-23 09:48:53 -05:00
markharwood f860e589a6 Test fix - GraphExploreResponseTests should not randomise array elements
Closes #33086
2018-08-23 15:00:30 +01:00
Dimitris Athanasiou 50441f97ae
HLRC: Add ML Get Buckets API (#33056)
Relates #29827
2018-08-23 09:35:06 +01:00
lipsill f84ed14294 Watcher: Improve error messages for CronEvalTool (#32800)
CronEvalTool prints an error only for cron expressions that result in
no upcoming time events.

If a cron expression results in less than the specified count
(default 10) time events, now all the coming times are printed
without displaying error message.

Closes #32735
2018-08-23 10:07:59 +02:00
Andrei Stefan de95dead2d
SQL: skip uppercasing/lowercasing function tests for AZ locales as well (#32910)
* Added the rest of the Locales that have different behavior for uppercasing/lowercasing scenarios to the skip list
2018-08-23 00:21:38 +03:00
Tal Levy dfc70ddcc7
rename pre-phase/pre-action to new/init (#32996)
this will keep things more consistent with the initial PhaseAfterStep, which has a phase name of `new`
2018-08-22 11:41:09 -07:00
Nik Everett 0cc99d270c
Switch ml basic tests to new style Requests (#32483)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/qa/ml-basic-multi-node` project to use
the new versions.
2018-08-22 14:23:43 -04:00
Nik Everett c3438bc8d8
Switch some watcher tests to new style Requests (#33044)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/qa/smoke-test-monitoring-with-watcher`,
`x-pack/qa/smoke-test-watcher`, and
`x-pack/qa/smoke-test-watcher-with-security` projects to use the new
versions.
2018-08-22 14:02:39 -04:00
Jason Tedor 528e727999
Fix method reference in comment in SchedulerEngine
This commit fixes the name of a method reference in a comment in
SchedulerEngine.
2018-08-22 10:20:20 -04:00
Jason Tedor 67bfb765ee
Refactor Netty4Utils#maybeDie (#33021)
In our Netty layer we have had to take extra precautions against Netty
catching throwables which prevents them from reaching the uncaught
exception handler. This code has taken on additional uses in NIO layer
and now in the scheduler engine because there are other components in
stack traces that could catch throwables and suppress them from reaching
the uncaught exception handler. This commit is a simple cleanup of the
iterative evolution of this code to refactor all uses into a single
method in ExceptionsHelper.
2018-08-22 10:18:07 -04:00
Benjamin Trent b02150a5ed
HLRC: close job refactor (#33031)
* HLRC: close job refactor

* Changing refactor to make job_id a string

* Changing set entity methodology
2018-08-22 06:54:08 -05:00
Alpar Torok 82d10b484a
Run forbidden api checks with runtimeJavaVersion (#32947)
Run forbidden APIs checks with runtime hava version
2018-08-22 09:05:22 +03:00
Ioannis Kakavas 0a4b55c9c0
[DOCS] Add RequestedAuthnContext Documentation (#32946)
Add documentation for #31238

- Add documentation for the req_authn_context_class_ref setting
- Add a section in SAML Guide regarding the use of SAML
  Authentication Context.
2018-08-22 08:37:50 +03:00
Yogesh Gaikwad 9f588c953f
[TEST] Split tests and skip file permission test on Windows (#32781)
Changes to split tests for keytab file test cases instead of
randomized testing for testing branches in the code in the
same test.
On windows platform, for keytab file permission test, we
required additional security permissions for the test
framework. As this was the only test that required those
permissions, skipping that test on windows platform.
The same scenario gets tested in *nix environments.

Closes#32768
2018-08-22 15:23:39 +10:00
Benjamin Trent e2ea83d217
HLRC: Add ML Get Job (#32960)
* HLRC: Adding GET ML Job info API

* HLRC: Adding GET Job ML API

* Fixing QueryPage license header

* Adding serialization tests, addressing minor issues

* Renaming querypage, changing the dependency on it

* Making things immutable

* Fixing build failure due to method rename
2018-08-21 21:02:28 -05:00
Tal Levy 55cb08a352
move ESLoggerFactory usage to LogManager (#33043)
Work done in #32513 has deprecated the old constructor in favor of
log4j2's LogManager
2018-08-21 17:59:32 -07:00
Nik Everett 2c81d7f77e
Build: Rework shadow plugin configuration (#32409)
This reworks how we configure the `shadow` plugin in the build. The major
change is that we no longer bundle dependencies in the `compile` configuration,
instead we bundle dependencies in the new `bundle` configuration. This feels
more right because it is a little more "opt in" rather than "opt out" and the
name of the `bundle` configuration is a little more obvious.

As an neat side effect of this, the `runtimeElements` configuration used when
one project depends on another now contains exactly the dependencies needed
to run the project so you no longer need to reference projects that use the
shadow plugin like this:

```
testCompile project(path: ':client:rest-high-level', configuration: 'shadow')
```

You can instead use the much more normal:

```
testCompile "org.elasticsearch.client:elasticsearch-rest-high-level-client:${version}"
```
2018-08-21 20:03:28 -04:00
Nik Everett f311680176 Monitoring: Clean up MonitoringIT
We recently reenabled MonitoringIT to hunt down #29880 but some of its
assertions were out of date. This updates the assertions.
2018-08-21 16:04:16 -04:00
Tal Levy 96869b253c
conditionally update CS only if StepInfo changes (#33004)
If we are waiting on a condition to be met, and the reason
it is not completed is unchanged, we find ourselves updating
cluster state over and over again and kicking of the ILM listeners
to re-check. This is overkill and can generate way too many
cluster state updates
2018-08-21 12:29:41 -07:00
Tal Levy 6780ab9d5c
add user authentication test for ILM (#32826) 2018-08-21 12:27:53 -07:00
Nik Everett fcf8cadd9a
Switch some x-pack tests to new style Requests (#32500)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/qa/audit-tests`,
`x-pack/qa/ml-disabled`, and `x-pack/qa/multi-node` projects to use the
new versions.
2018-08-21 14:48:53 -04:00
Jason Tedor 28d12b05b7
Move ML tests to be sub-projects of ML (#33026)
This commit moves the ML QA tests to be a sub-project of ML. The purpose
of this refactoring is to enable ML developers to run
:x-pack:plugin:ml:check and run the vast majority of a ML tests with a
single command (this still does not contain the ML REST tests, nor the
upgrade tests). This simplifies local development for faster iteration.
2018-08-21 12:23:21 -04:00
Adrien Grand 5b446b81ef Reenable MonitoringIT#testMonitoringService.
When discussing this test, it made little sense that testMonitoringService
would fail but not testMonitoringBulk given their similarity. So we argeed to
enable it again.

Relates #29880
2018-08-21 16:53:59 +02:00
Colin Goodheart-Smithe 75cae4560c
fix compile error after merge 2018-08-21 14:47:54 +01:00
Ioannis Kakavas 1b583978e9
[DOCS] Add FIPS 140-2 documentation (#32928)
* Add relevant documentation for FIPS 140-2 compliance.
* Introduce `fips_mode` setting.
* Discuss necessary configuration for FIPS 140-2
* Discuss introduced limitations by FIPS 140-2
2018-08-21 16:20:00 +03:00
Benjamin Trent 3f91bbfa6b
[ML] Allowing _close to accept body payloads for options (#32989) (#33000) 2018-08-21 08:08:26 -05:00
markharwood 38bdf9ce32
HLRC GraphClient and associated tests (#32366)
GraphClient for the high level REST client and associated tests.
Part of #29827 work
2018-08-21 13:29:18 +01:00
Colin Goodheart-Smithe 10c60fae93
Merge branch 'master' into index-lifecycle 2018-08-21 11:54:06 +01:00
Colin Goodheart-Smithe 81c9e8d939
make operation mode YAML test more robust 2018-08-21 11:50:40 +01:00
Ioannis Kakavas 65d4f27873
[DOCS] Add configurable password hashing docs (#32849)
* [DOCS] Add configurable password hashing docs

Adds documentation about the newly introduced configuration option
for setting the password hashing algorithm to be used for the users
cache and for storing credentials for the native and file realm.
2018-08-21 12:05:42 +03:00
Tim Vernum b595b1a20c
Handle 6.4.0+ BWC for Application Privileges (#32929)
When the application privileges feature was backported to 6.x/6.4 the
BWC version checks on the backport were updated to 6.4.0, but master
was not updated.

This commit updates all relevant version checks, and adds tests.
2018-08-21 17:58:37 +10:00
Yogesh Gaikwad 38886e8f23
[DOCS] Add Kerberos troubleshooting documentation (#32803)
This commit adds troubleshooting section for Kerberos.
Most of the times the problems seen are caused due to invalid
configurations like keytab missing principals or credentials
not up to date. Time synchronization is an important part for
Kerberos infrastructure and the time skew can cause problems.
To debug further documentation explains how to enable JAAS
Kerberos login module debugging and Kerberos/SPNEGO debugging
by setting JVM system properties.
2018-08-21 16:30:28 +10:00
Jason Tedor 6d62d6755a
Fix typo in comment in scheduler engine
This commit fixes a minor typo in a big block comment in
SchedulerEngine.java.
2018-08-20 22:58:07 -04:00
Jason Tedor ad0a965db9
Protect scheduler engine against throwing listeners (#32998)
There are two problems with the scheduler engine today. Both relate to
listeners that throw.

The first problem is that any triggered listener that throws a plain old
exception will cause no additional listeners to be triggered for the
event, and will also cause the scheduler to never be invoked again. This
leads to lost events and is bad.

The second problem is that any triggered listener that throws an error
of the fatal kind will not lead to that error because caught by the
uncaught exception handler. This is because the triggered listener is
executed as a future task under a scheduled thread pool executor. A
throwable there goes caught by the JDK framework and set as the outcome
on the future task. Since we never inspect these tasks for their
outcomes, nor is there a good place to do this, we have to handle these
errors ourselves. To do this, we catch them and dispatch them to the
uncaught exception handler via a forked thread. This is similar to our
handling in Netty.
2018-08-20 22:07:16 -04:00
Tal Levy 2b3cb240f2 add INDEX_LIFECYCLE_ORIGIN to run as xpack 2018-08-20 15:44:47 -07:00
Lisa Cawley 2feda8aae0
[DOC] Splits role mapping APIs into separate pages (#32797) 2018-08-20 14:30:42 -07:00
Benjamin Trent 3fbaae10af
HLRC: ML Close Job (#32943)
* HLRC: Adding ML Close Job API

HLRC: Adding ML Close Job API

* reconciling request converters

* Adding serialization tests and addressing PR comments

* Changing constructor order
2018-08-20 16:05:56 -05:00
Jason Tedor 9050c7e846
Generalize remote license checker (#32971)
Machine learning has baked a remote license checker for use in checking
license compatibility of a remote license. This remote license checker
has general usage for any feature that relies on a remote cluster. For
example, cross-cluster replication will pull changes from a remote
cluster and require that the local and remote clusters have platinum
licenses. This commit generalizes the remote cluster license check for
use in cross-cluster replication.
2018-08-20 15:33:29 -04:00
Nik Everett 462e91d362
Logging: Use settings when building daemon threads (#32751)
Subclasses of `EsIntegTestCase` run multiple Elasticsearch nodes in the
same JVM and when we log we look at the name of the thread to figure out
the node name. This makes sure that all calls to `daemonThreadFactory`
include the node name.

Closes #32574

I'd like to follow this up with more drastic changes that make it
impossible to do this incorrectly but that change is much larger than
this and I'd like to get these log lines fixed up sooner rather than
later.
2018-08-20 13:53:15 -04:00
Tal Levy 5ce082cd0a
copy LifecyclePolicy to protocol.xpack (#32915)
This is the final PR for copying over the necessary components for
clients to parse/render LifecyclePolicy. Changes include:

- move of named-x-content server objects away from client
- move validation into the client copy of LifecyclePolicy
- move LifecycleAction into an interface with `getName`
2018-08-20 08:32:22 -07:00
Ioannis Kakavas eef0e35913
Add mzn and dz to unsupported locales (#32957)
Add mzn and dz to the list of unsupported locales
for Kerberos tests.
2018-08-20 17:12:02 +03:00
Ioannis Kakavas 6905ca9d6c
Use settings from the context in BootstrapChecks (#32908)
Use settings from the context in BootstrapChecks
instead of passing them in the constructor
2018-08-20 17:01:10 +03:00
Colin Goodheart-Smithe 3736097e19
Merge branch 'master' into index-lifecycle 2018-08-20 10:06:02 +01:00
Yogesh Gaikwad e143cce865
[Kerberos] Add documentation for Kerberos realm (#32662)
This commit adds documentation for configuring Kerberos realm.
Configuring Kerberos realm documentation highlights important
terminology and requirements before creating Kerberos realm.
Most of the documentation is centered around configuration from
Elasticsearch rather than go deep into Kerberos implementation.
Kerberos realm settings are mentioned in the security settings
for Kerberos realm.
2018-08-20 17:23:14 +10:00
Alexander Reelsen 3fa36807f8
Watcher: Properly find next valid date in cron expressions (#32734)
When a list/an array of cron expressions is provided, and one of those addresses
is already expired, the expired one will be considered as an option
instead of the valid next one.

This commit also reduces the visibility of the CronnableSchedule and
refactors a comparator to look like java 8.
2018-08-20 09:18:51 +02:00
Alpar Torok 4b34b3f4aa
Set forbidden APIs target compatibility to compiler java version (#32935)
Set forbidden apis target compatibility to compiler version

Fix outstanding deprecation
2018-08-20 09:27:02 +03:00
Tim Brooks de92d2ef1f
Move connection listener to ConnectionManager (#32956)
This is a followup to #31886. After that commit the
TransportConnectionListener had to be propogated to both the
Transport and the ConnectionManager. This commit moves that listener
to completely live in the ConnectionManager. The request and response
related methods are moved to a TransportMessageListener. That listener
continues to live in the Transport class.
2018-08-18 10:09:24 -06:00