Commit Graph

644 Commits

Author SHA1 Message Date
Dimitris Athanasiou 9600819cef
[HLRC][ML] Add ML delete datafeed API to HLRC (#33667)
Relates #29827
2018-09-13 17:13:36 +01:00
Dimitris Athanasiou 2eb2313b60
[HLRC][ML] Add ML put datafeed API to HLRC (#33603)
This also changes both `DatafeedConfig` and `DatafeedUpdate`
to store the query and aggs as a bytes reference. This allows
the client to remove its dependency to the named objects
registry of the search module.

Relates #29827
2018-09-12 14:52:36 +01:00
Benjamin Trent 27e07ec859
HLRC: ML Delete Forecast API (#33526)
* HLRC: ML Delete Forecast API
2018-09-11 16:32:52 -07:00
Ed Savage 2f3b542d57
HLRC: Add ML get categories API (#33465)
HLRC: Adding the ML 'get categories' API
2018-09-11 12:48:14 +01:00
Julie Tibshirani 95100e05f9
Remove some duplicate request conversion methods. (#33538) 2018-09-09 20:02:35 -07:00
S.Y. Wang 9073dbefd6 HLRC: Add put stored script support to high-level rest client (#31323)
Relates to #27205
2018-09-09 13:47:47 +02:00
Michael Basnight d6b40f4c7b
HLRC: split ingest request converters (#33435)
In an effort to encapsulate the different clients, the request
converters are being shuffled around. This splits the IngestClient
request converters.
2018-09-07 13:44:16 -05:00
Michael Basnight 8d61457d9c
HLRC: split xpack request converters (#33444)
In an effort to encapsulate the different clients, the request
converters are being shuffled around. This splits the XPackClient
request converters.
2018-09-07 13:30:57 -05:00
Michael Basnight 43592305a1
HLRC: split watcher request converters (#33442)
In an effort to encapsulate the different clients, the request
converters are being shuffled around. This splits the WatcherClient
request converters.
2018-09-07 13:08:31 -05:00
Jay Modi 9d16a7b7f0
HLRC: add enable and disable user API support (#33481)
This change adds support for enable and disable user APIs to the high
level rest client. There is a common request base class for both
requests with specific requests that simplify the use of these APIs.

The response for these APIs is simply an empty object so a new response
class has been created for cases where we expect an empty response to
be returned.

Finally, the put user documentation has been moved to the proper
location that is not within an x-pack sub directory and the document
tags no longer contain x-pack.

See #29827
2018-09-07 11:51:37 -06:00
Benjamin Trent 4d233107f8
HLRC: ML Forecast Job (#33506)
* HLRC: ML Forecast job
2018-09-07 11:16:58 -05:00
Michael Basnight 42469a9930
HLRC: split migration request converters (#33436)
In an effort to encapsulate the different clients, the request
converters are being shuffled around. This splits the MigrationClient
request converters.
2018-09-07 09:46:27 -05:00
Michael Basnight c32e71c61f
HLRC: split snapshot request converters (#33439)
In an effort to encapsulate the different clients, the request
converters are being shuffled around. This splits the SnapshotClient
request converters.
2018-09-07 09:25:47 -05:00
Benjamin Trent 9230a48722
HLRC: ML Post Data (#33443)
* HLRC: ML Post data
2018-09-07 07:04:27 -05:00
Jim Ferenczi 79cd6385fe
Collapse package structure for metrics aggs (#33463)
This change collapses all metrics aggregations classes into a single package `org.elasticsearch.aggregations.metrics`.
It also restricts the visibility of some classes (aggregators and factories) that should not be used outside of the package.

Relates #22868
2018-09-07 10:58:06 +02:00
Michael Basnight 6a3adbd935
HLRC: split tasks request converters (#33441)
In an effort to encapsulate the different clients, the request
converters are being shuffled around. This splits the TasksClient
request converters.
2018-09-06 14:11:27 -05:00
Michael Basnight 8ce4ceb59e
HLRC: split license request converters (#33438)
In an effort to encapsulate the different clients, the request
converters are being shuffled around. This splits the LicenseClient
request converters.
2018-09-06 11:37:27 -05:00
Michael Basnight 5251300fb5
HLRC: split graph request converters (#33429)
In an effort to encapsulate the different clients, the request
converters are being shuffled around. This splits the GraphClient
request converters.
2018-09-06 11:33:26 -05:00
Jim Ferenczi 7ad71f906a
Upgrade to a Lucene 8 snapshot (#33310)
The main benefit of the upgrade for users is the search optimization for top scored documents when the total hit count is not needed. However this optimization is not activated in this change, there is another issue opened to discuss how it should be integrated smoothly.
Some comments about the change:
* Tests that can produce negative scores have been adapted but we need to forbid them completely: #33309

Closes #32899
2018-09-06 14:42:06 +02:00
Benjamin Trent 9b6bbc0182
HLRC: ML Update Job (#33392)
* HLRC: ML Update Job
2018-09-06 07:18:09 -05:00
Jay Modi ea52277a1e
HLRest: add put user API (#32332)
This commit adds a security client to the high level rest client, which
includes an implementation for the put user api. As part of these
changes, a new request and response class have been added that are
specific to the high level rest client. One change here is that the response
was previously wrapped inside a user object. The plan is to remove this
wrapping and this PR adds an unwrapped response outside of the user
object so we can remove the user object later on.

See #29827
2018-09-05 10:56:30 -06:00
Michael Basnight 7319bc7411
HLRC: split cluster request converters (#33400)
In an effort to encapsulate the different clients, the request
converters are being shuffled around. This splits the ClusterClient
request converters.
2018-09-05 09:34:47 -05:00
Dimitris Athanasiou 24776b2b80
HLRC: Add ML get influencers API (#33389)
Relates #29827
2018-09-05 15:06:55 +01:00
Dimitris Athanasiou bd8d82d859 [ML] Add missing param description in GetOverallBucketsRequest 2018-09-04 17:31:49 +01:00
Sohaib Iftikhar 761e8c461f HLRC: Add delete by query API (#32782)
Adds the delete-by-query API to the High Level REST Client.
2018-09-04 08:56:26 -04:00
Alpar Torok 7f7e8fd733
Disable assemble task instead of removing it (#33348) 2018-09-04 07:32:14 +03:00
Dimitris Athanasiou 54fe7fb5a1
HLRC: Add ML get overall buckets API (#33297)
Relates #29827
2018-09-03 13:34:36 +01:00
Sohaib Iftikhar 389bf67275 HLREST: add update by query API (#32760)
Adds update by query to the high level rest client.
2018-09-02 15:15:00 -04:00
Benjamin Trent 6770a456b8
HLRC: ML Flush job (#33187)
* HLRC: ML Flush job

* Fixing package, paths, and test

* Addressing comments
2018-09-01 16:01:23 -05:00
Benjamin Trent 19b14fa5ed
HLRC: Adding ML Job stats (#33183)
* HLRC: Adding pojos for get job stats

HLRC: Adding pojos for job stats request

* HLRC: Adding job stats pojos

* HLRC: ML job stats

* Minor syntax changes and adding license headers

* minor comment change

* Moving to client package, minor changes

* Addressing PR comments

* removing bad sleep

* addressing minor comment around test methods

* adding toplevel random fields for tests

* addressing minor review comments
2018-09-01 13:32:18 -05:00
Nik Everett f28cddf951
LLREST: Drop deprecated methods (#33223)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. In a
long series of PRs I've changed all of the old style requests. This
drops the deprecated methods and will be released with 7.0.
2018-09-01 11:11:25 -04:00
Jay Modi f063587083
HLRC: add client side RefreshPolicy (#33209)
With the switch to client side request and response objects, we need a
client side version of RefreshPolicy. This change adds a client side
version of RefreshPolicy along with a method to add it to the
parameters of a request. The existing method to add
WriteRequest.RefreshPolicy to the parameters of a request is now
deprecated.
2018-08-29 21:44:33 -06:00
Dimitris Athanasiou a5b34c75b0
HLRC: Add ML Get Records API (#33085)
Relates #29827
2018-08-29 15:03:58 +01:00
Michael Basnight 1be3dd5504
HLRC: create base timed request class (#33216)
There are many requests that allow the user to set a few timeouts
on. This class will allow requests impl'd in HLRC to extend from, and
allow users to set those values without significant work to add them to
every request.
2018-08-28 17:23:41 -05:00
Dimitris Athanasiou 5697d93cbf HLRC: Move ML protocol classes into client ml package (#33203)
* HLRC: Move ML protocol classes into client ml package

* Do not use log4j deprecation handler

* JavaDoc should refer to correct Job path
2018-08-28 14:53:31 -05:00
Sohaib Iftikhar 7f5e29ddb2 HLREST: add reindex API (#32679)
Adds the reindex API to the high level REST client.
2018-08-28 13:02:23 -04:00
Michael Basnight 5f0f990afd
HLRC: Use Optional in validation logic (#33104)
The Validatable class comes from an old class in server code, that
assumed null was returned in the event of validation having no
errors. This commit changes that to use Optional, which is cleaner than
passing around null objects.
2018-08-28 10:33:18 -05:00
Armin Braun daee8bd133
HLRC+MINOR: Remove Unused Private Method (#33165)
* This one seems to be unused since 92eb32477
2018-08-28 07:44:01 +02:00
Albert Zaharovits fbe609d589
Reload Secure Settings REST specs & docs (#32990)
This is a minimal REST API spec and docs for the REST handler
for the `_nodes/reload_secure_settings endpoint`.

Relates #29135
2018-08-26 14:49:32 +03:00
Nik Everett dd4a8dc444
Switch remaining tests to new style Requests (#33109)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `client` and `distribution` projects to use
the new versions.
2018-08-24 16:37:25 -04:00
Armin Braun 61f5c188e0 HLRC: Fix Compile Error From Missing Throws (#33083)
* 50441f97ae745814db96c262e99d0f465aca5b2c#diff-53a95fe7ded21313483f1b2f15977395L72 removed the throws breaking compilation here
2018-08-23 11:24:32 +01:00
Dimitris Athanasiou 50441f97ae
HLRC: Add ML Get Buckets API (#33056)
Relates #29827
2018-08-23 09:35:06 +01:00
Luca Cavanna 393eec1482
Set maxScore for empty TopDocs to Nan rather than 0 (#32938)
We used to set `maxScore` to `0` within `TopDocs` in situations where there is really no score as the size was set to `0` and scores were not even tracked. In such scenarios, `Float.Nan` is more appropriate, which gets converted to `max_score: null` on the REST layer. That's also more consistent with lucene which set `maxScore` to `Float.Nan` when merging empty `TopDocs` (see `TopDocs#merge`).
2018-08-22 17:23:54 +02:00
Michael Basnight e9912081dd
HLRC: Create server agnostic request and response (#32912)
The HLRC has historically reused the same Request and Response classes
that the server module uses. This commit deprecates the use of any
server module Request and Response classes, and adds a small bit of
validation logic that differs from server slightly, in that it does not
assume a check for a null ValidationException class is not enough to
determine if validation failed.
2018-08-22 09:19:58 -05:00
Benjamin Trent b02150a5ed
HLRC: close job refactor (#33031)
* HLRC: close job refactor

* Changing refactor to make job_id a string

* Changing set entity methodology
2018-08-22 06:54:08 -05:00
Alpar Torok 82d10b484a
Run forbidden api checks with runtimeJavaVersion (#32947)
Run forbidden APIs checks with runtime hava version
2018-08-22 09:05:22 +03:00
Benjamin Trent e2ea83d217
HLRC: Add ML Get Job (#32960)
* HLRC: Adding GET ML Job info API

* HLRC: Adding GET Job ML API

* Fixing QueryPage license header

* Adding serialization tests, addressing minor issues

* Renaming querypage, changing the dependency on it

* Making things immutable

* Fixing build failure due to method rename
2018-08-21 21:02:28 -05:00
Nik Everett 2c81d7f77e
Build: Rework shadow plugin configuration (#32409)
This reworks how we configure the `shadow` plugin in the build. The major
change is that we no longer bundle dependencies in the `compile` configuration,
instead we bundle dependencies in the new `bundle` configuration. This feels
more right because it is a little more "opt in" rather than "opt out" and the
name of the `bundle` configuration is a little more obvious.

As an neat side effect of this, the `runtimeElements` configuration used when
one project depends on another now contains exactly the dependencies needed
to run the project so you no longer need to reference projects that use the
shadow plugin like this:

```
testCompile project(path: ':client:rest-high-level', configuration: 'shadow')
```

You can instead use the much more normal:

```
testCompile "org.elasticsearch.client:elasticsearch-rest-high-level-client:${version}"
```
2018-08-21 20:03:28 -04:00
Dimitris Athanasiou 28a0df2c7f
HLRC: Clear ML data after client tests (#33023)
This commit duplicates the `MlRestTestStateCleaner` to make sure
all ML data is removed after each test. After implementing
the job and datafeed APIs in the HLRC, we shall replace this
implementation with one using the HLRC itself.

Closes #32993
2018-08-21 18:12:28 +01:00
markharwood 38bdf9ce32
HLRC GraphClient and associated tests (#32366)
GraphClient for the high level REST client and associated tests.
Part of #29827 work
2018-08-21 13:29:18 +01:00
Benjamin Trent 3fbaae10af
HLRC: ML Close Job (#32943)
* HLRC: Adding ML Close Job API

HLRC: Adding ML Close Job API

* reconciling request converters

* Adding serialization tests and addressing PR comments

* Changing constructor order
2018-08-20 16:05:56 -05:00
Nik Everett 34295fad87 HLREST: AwaitsFix ML Test
It leaks state into other tests causing them to fail sometimes.

Relates to #32993
2018-08-20 13:05:55 -04:00
Nik Everett f853f6f03c
HLRC: Forbid all Elasticsearch logging infra (#32784)
All of the Elasticsearch logging infrastructure relies on log4j but we
don't want the high level rest client to rely on log4j2. All of its
logging goes through commons-logging because our dependencies drag in
commons logging already. Anyway, this bans direct use of Elasticsearch's
logging infrastructure in the high level REST client. It is still
possible to use it indirectly though and there isn't anything we can
really do about that until we split the high level rest client from
Elasticsearch's server jar.
2018-08-20 08:55:24 -04:00
Dimitris Athanasiou 76aba8ad7b
HLRC: Move ML request converters into their own class (#32906) 2018-08-17 10:57:00 +01:00
Benjamin Trent aedc2c1c49
HLRC: adding machine learning delete job (#32820)
* HLRC: adding machine learning delete job

* Fixing whitespace

* Moving docs and tests around

* Unifying ml asciidoc file naming convention
2018-08-16 07:18:43 -05:00
Benjamin Trent f90f06ede1
HLRC: adding machine learning open job (#32860)
* HLRC: adding machine learning open job

HLRC: adding ML open job API call

* Addressing @droberts195 comments and fixing minor style issue
2018-08-15 13:19:41 -05:00
Lee Hinman 48281ac5bc
Use generic AcknowledgedResponse instead of extended classes (#32859)
This removes custom Response classes that extend `AcknowledgedResponse` and do nothing, these classes are not needed and we can directly use the non-abstract super-class instead.

While this appears to be a large PR, no code has actually changed, only class names have been changed and entire classes removed.
2018-08-15 08:06:14 -06:00
markharwood eb812c8b7d Removed flaky test. Looks like randomisation makes these assertions unreliable.
This test is superfluous - it was added to address #32770 but it later turned out there was an existing test that just required a fix to provide the missing test coverage.

Closes #32855
2018-08-15 10:06:39 +01:00
David Roberts c985f500f4
[ML] Add high level REST client docs for ML put job endpoint (#32843)
Relates #29827
Relates #32726
2018-08-14 19:52:54 +01:00
Andy Bristol 84aa87f638 [test] mute testSearchWithSignificantTermsAgg
For #32855
2018-08-14 10:59:44 -07:00
markharwood e5ab09f708
Aggregations/HL Rest client fix: missing scores (#32774)
Significance score doubles were being parsed as long. Existing tests did not catch this because SignificantLongTermsTests and SignificantStringTermsTests did not set the score. Fixed these and also added integration test.

Thanks for the report/fix, Blakko

Closes #32770
2018-08-14 11:14:47 +01:00
Yannick Welsch 00b006f033
HLRC: Add Delete License API (#32586)
Relates to #29827
2018-08-14 11:55:54 +02:00
Luca Cavanna 3e692c3f3d
HLRC: migration get assistance API (#32744)
The request and response classes have been extracted from `IndexUpgradeInfoAction` into top-level classes, and moved to the protocol jar. The `UpgradeActionRequired` enum is also moved.

Relates to #29827
2018-08-13 11:05:27 +02:00
Nik Everett 2df80ca7e3
HLRC: Ban LoggingDeprecationHandler (#32756)
LoggingDeprecationHandler requires log4j2 but we don't require log4j2 in
the client. This bans LoggingDeprecationHandler and removes all uses of
it in the high level client.

Closes #32151
2018-08-10 10:17:48 -04:00
Benjamin Trent 94a9b253db
Adding ML HLRC wrapper and put_job API call (#32726)
* Adding ML HLRC wrapper and put_job API call

* Changing integration test job to have consistent stucture
2018-08-10 07:16:55 -05:00
Ryan Ernst 9a16491ebf
Build: Add back setting artifact id of pom for rest high level client (#32731)
This commit adds back the publishing section that sets the artifact id
of the generated pom file for the high level rest client. This was
accidentally removed during a consolidationo of the shadow plugin logic.
2018-08-09 10:11:08 -07:00
Lee Hinman 7af28c48c3
Switch WritePipelineResponse to AcknowledgedResponse (#32722)
We previously discussed moving the classes extending `AcknowledgedResponse` to
simply use `AcknowledgedResponse`, making the class non-abstract.

This moves the first class to do this, removing `WritePipelineResponse` in the
process.

If we like the way this looks, I will switch the remaining classes over to using
`AcknowledgedResponse`.
2018-08-08 16:21:58 -06:00
Armin Braun 580d59e2d7
CORE: Upgrade to Jackson 2.8.11 (#32670)
* closes #30352
2018-08-08 12:04:25 +02:00
Andy Bristol 8bfb0f3f8d
serialize suggestion responses as named writeables (#30284)
Suggestion responses were previously serialized as streamables which
made writing suggesters in plugins with custom suggestion response types
impossible. This commit makes them serialized as named writeables and
provides a facility for registering a reader for suggestion responses
when registering a suggester.

This also makes Suggestion responses abstract, requiring a suggester
implementation to provide its own types. Suggesters which do not need
anything additional to what is defined in Suggest.Suggestion should
provide a minimal subclass.

The existing plugin suggester integration tests are removed and
replaced with an equivalent implementation as an example
plugin.
2018-08-07 13:31:00 -07:00
Igor Motov e641fccfe3
Rest HL client: Add get license action (#32438)
Rest HL client: Add get license action

Continues to use String instead of a more complex License class to
hold the license text similarly to put license.

Relates #29827
2018-08-06 07:15:40 -07:00
Igor Motov ada80d7fc8
Suppress LicensingDocumentationIT.testPutLicense in release builds (#32613)
The testPutLicense test tries to put a license generated using
snapshot keys into release cluster. This commit suppresses the
test during the release builds.

Closes #32580
2018-08-03 11:59:51 -07:00
Michael Basnight 9dcf3f5aee
HLRC: Move commercial clients from XPackClient (#32596)
The commercial clients were improperly placed into XPackClient, which is
a wrapper for the miscellaneous usage and info APIs. This commit moves
them into the HLRC.
2018-08-02 19:38:49 -05:00
Colin Goodheart-Smithe 097c4287f3
Mutes LicensingDocumentationIT due to #32580 2018-08-02 11:58:51 +01:00
Nik Everett 21eb9695af
Build: Remove shadowing from benchmarks (#32475)
Removes shadowing from the benchmarks. It isn't *strictly* needed. We do
have to rework the documentation on how to run the benchmark, but it
still seems to work if you run everything through gradle.
2018-07-31 17:31:13 -04:00
Nik Everett 22459576d7
Logging: Make node name consistent in logger (#31588)
First, some background: we have 15 different methods to get a logger in
Elasticsearch but they can be broken down into three broad categories
based on what information is provided when building the logger.

Just a class like:
```
private static final Logger logger = ESLoggerFactory.getLogger(ActionModule.class);
```
or:
```
protected final Logger logger = Loggers.getLogger(getClass());
```

The class and settings:
```
this.logger = Loggers.getLogger(getClass(), settings);
```

Or more information like:
```
Loggers.getLogger("index.store.deletes", settings, shardId)
```

The goal of the "class and settings" variant is to attach the node name
to the logger. Because we don't always have the settings available, we
often use the "just a class" variant and get loggers without node names
attached. There isn't any real consistency here. Some loggers get the
node name because it is convenient and some do not.

This change makes the node name available to all loggers all the time.
Almost. There are some caveats are testing that I'll get to. But in
*production* code the node name is node available to all loggers. This
means we can stop using the "class and settings" variants to fetch
loggers which was the real goal here, but a pleasant side effect is that
the ndoe name is now consitent on every log line and optional by editing
the logging pattern. This is all powered by setting the node name
statically on a logging formatter very early in initialization.

Now to tests: tests can't set the node name statically because
subclasses of `ESIntegTestCase` run many nodes in the same jvm, even in
the same class loader. Also, lots of tests don't run with a real node so
they don't *have* a node name at all. To support multiple nodes in the
same JVM tests suss out the node name from the thread name which works
surprisingly well and easy to test in a nice way. For those threads
that are not part of an `ESIntegTestCase` node we stick whatever useful
information we can get form the thread name in the place of the node
name. This allows us to keep the logger format consistent.
2018-07-31 10:54:24 -04:00
Yannick Welsch 11f6983d69
HLRC: Add delete watch action (#32337)
Adds the "delete watch" API to the High-Level Rest Client.

Relates #29827
2018-07-31 10:29:22 +02:00
Igor Motov 29c802f88e
Rest HL client: Add put license action (#32214)
In the HL REST client we replace the License object with a string, because of 
complexity of this class. It is also not really needed on the client side since 
end-users are not interacting with the license besides passing it as a string 
to the server.

Relates #29827
2018-07-24 11:19:07 -07:00
Daniel Mitterdorfer 73a38895fd
Add Restore Snapshot High Level REST API
With this commit we add the restore snapshot API to the Java high level
REST client.

Relates #27205
Relates #32155
2018-07-24 16:17:09 +02:00
Christoph Büscher 59cf600e03
Register ERR metric with NamedXContentRegistry (#32320)
This adds the ERR metric to the provided xContent parsers in the module and the
high level rest client registry. Also adding integration tests to make sure the
metric is correctly registered and usable from the client.
2018-07-24 16:05:43 +02:00
Christoph Büscher fe6bb75eb4
Rename ranking evaluation `quality_level` to `metric_score` (#32168)
The notion of "quality" is an overloaded term in the search ranking evaluation 
context. Its usually used to decribe certain levels of "good" vs. "bad" of a 
seach result with respect to the users information need. We currently report the 
result of the ranking evaluation as `quality_level` which is a bit missleading.
This changes the response parameter name to `metric_score` which fits better.
2018-07-23 22:25:02 +02:00
Christoph Büscher ff87b7aba4
Remove unnecessary warning supressions (#32250) 2018-07-23 11:31:04 +02:00
Luca Cavanna 1390a849e1
[TEST] improve REST high-level client naming conventions check (#32244)
Check the deprecated methods are effectively deprecated.
Also compare the class rather than their names when checking argument types.
2018-07-21 10:07:08 +02:00
Christoph Büscher 5cbd9ad177
Rename ranking evaluation response section (#32166)
Currently the ranking evaluation response contains a 'unknown_docs' section 
for each search use case in the evaluation set. It contains document ids for 
results in the search hits that currently don't have a quality rating.
This change renames it to `unrated_docs`, which better reflects its purpose.
2018-07-20 11:43:46 +02:00
Alexander Reelsen 202894b832
Rest HL client: Add put watch action (#32026)
Relates #29827

This implementation behaves like the current transport client, that you basically cannot configure a Watch POJO representation as an argument to the put watch API, but only a bytes reference. You can use the the `WatchSourceBuilder` from the `org.elasticsearch.plugin:x-pack-core` dependency to build watches.

This commit also changes the license type to trial, so that watcher is available in high level rest client tests.

/cc @hub-cap
2018-07-19 10:40:54 +02:00
Alpar Torok 38e2e1d553
Detect and prevent configuration that triggers a Gradle bug (#31912)
* Detect and prevent configuration that triggers a Gradle bug

As we found in #31862, this can lead to a lot of wasted time as it's not
immediatly obvius what's going on.
Givent how many projects we have it's getting increasingly easier to run
into gradle/gradle#847.
2018-07-19 06:46:58 +00:00
Nik Everett 1b97652a4c
Build: Move shadow customizations into common code (#32014)
Moves the customizations to the build to produce nice shadow jars and
javadocs into common build code, mostly BuildPlugin with a little into
the root build.gradle file. This means that any project that applies the
shadow plugin will automatically be set up just like the high level rest
client:
* The non-shadow jar will not be built
* The shadow jar will not have a "classifier"
* Tests will run against the shadow jar
* Javadoc will include all of the shadowed classes
* Service files in `META-INF/services` will be merged
2018-07-17 14:20:41 -04:00
Luca Cavanna 049966a829
Check that client methods match API defined in the REST spec (#31825)
We have been encountering name mismatches between API defined in our
REST spec and method names that have been added to the high-level REST
client. We should check this automatically to prevent furher mismatches,
and correct all the current ones.

This commit adds a test for this and corrects the issues found by it.
2018-07-17 11:26:28 +02:00
Ioannis Kakavas 9e529d9d58
Enable testing in FIPS140 JVM (#31666)
Ensure our tests can run in a FIPS JVM

JKS keystores cannot be used in a FIPS JVM as attempting to use one
in order to init a KeyManagerFactory or a TrustManagerFactory is not
allowed.( JKS keystore algorithms for private key encryption are not
FIPS 140 approved)
This commit replaces JKS keystores in our tests with the
corresponding PEM encoded key and certificates both for key and trust
configurations.
Whenever it's not possible to refactor the test, i.e. when we are
testing that we can load a JKS keystore, etc. we attempt to
mute the test when we are running in FIPS 140 JVM. Testing for the
JVM is naive and is based on the name of the security provider as
we would control the testing infrastrtucture and so this would be
reliable enough.
Other cases of tests being muted are the ones that involve custom
TrustStoreManagers or KeyStoreManagers, null TLS Ciphers and the
SAMLAuthneticator class as we cannot sign XML documents in the
way we were doing. SAMLAuthenticator tests in a FIPS JVM can be
reenabled with precomputed and signed SAML messages at a later stage.

IT will be covered in a subsequent PR
2018-07-17 10:54:10 +03:00
Yogesh Gaikwad 6717df3c2d
[Test] Modify assert statement for ssl handshake (#32072)
There have been changes in error messages for `SSLHandshakeException`.
This has caused a couple of failures in our tests.
This commit modifies test verification to assert on exception type of
class `SSLHandshakeException`.
There was another issue in Java11 which caused NPE. The bug has now
been fixed on Java11 - early access build 22.
Bug Ref: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8206355
Enable the skipped tests due to this bug.

Closes #31940
2018-07-17 15:43:03 +10:00
Christoph Büscher 3587d8872e
[Tests] Fix failure due to changes exception message (#32036)
Java 11 seems to get more verbose on the ClassCastException we check for in
SearchDocumentationIT. This changes the test from asserting the exact exception
message to only checking the two classes involved are part of the message.

Closes #32029
2018-07-16 11:22:42 +02:00
Jack Conradson 42ca520377
Clean Up Snapshot Create Rest API (#31779)
Make SnapshotInfo and CreateSnapshotResponse parsers lenient for backwards compatibility.  Remove extraneous fields from CreateSnapshotRequest toXContent.
2018-07-13 13:07:26 -07:00
Colin Goodheart-Smithe 0edb096eb4 Adds a new auto-interval date histogram (#28993)
* Adds a new auto-interval date histogram

This change adds a new type of histogram aggregation called `auto_date_histogram` where you can specify the target number of buckets you require and it will find an appropriate interval for the returned buckets. The aggregation works by first collecting documents in buckets at second interval, when it has created more than the target number of buckets it merges these buckets into minute interval bucket and continues collecting until it reaches the target number of buckets again. It will keep merging buckets when it exceeds the target until either collection is finished or the highest interval (currently years) is reached. A similar process happens at reduce time.

This aggregation intentionally does not support min_doc_count, offest and extended_bounds to keep the already complex logic from becoming more complex. The aggregation accepts sub-aggregations but will always operate in `breadth_first` mode deferring the computation of sub-aggregations until the final buckets from the shard are known. min_doc_count is effectively hard-coded to zero meaning that we will insert empty buckets where necessary.

Closes #9572

* Adds documentation

* Added sub aggregator test

* Fixes failing docs test

* Brings branch up to date with master changes

* trying to get tests to pass again

* Fixes multiBucketConsumer accounting

* Collects more buckets than needed on shards

This gives us more options at reduce time in terms of how we do the
final merge of the buckeets to produce the final result

* Revert "Collects more buckets than needed on shards"

This reverts commit 993c782d117892af9a3c86a51921cdee630a3ac5.

* Adds ability to merge within a rounding

* Fixes nonn-timezone doc test failure

* Fix time zone tests

* iterates on tests

* Adds test case and documentation changes

Added some notes in the documentation about the intervals that can bbe
returned.

Also added a test case that utilises the merging of conseecutive buckets

* Fixes performance bug

The bug meant that getAppropriate rounding look a huge amount of time
if the range of the data was large but also sparsely populated. In
these situations the rounding would be very low so iterating through
the rounding values from the min key to the max keey look a long time
(~120 seconds in one test).

The solution is to add a rough estimate first which chooses the
rounding based just on the long values of the min and max keeys alone
but selects the rounding one lower than the one it thinks is
appropriate so the accurate method can choose the final rounding taking
into account the fact that intervals are not always fixed length.

Thee commit also adds more tests

* Changes to only do complex reduction on final reduce

* merge latest with master

* correct tests and add a new test case for 10k buckets

* refactor to perform bucket number check in innerBuild

* correctly derive bucket setting, update tests to increase bucket threshold

* fix checkstyle

* address code review comments

* add documentation for default buckets

* fix typo
2018-07-13 13:08:35 -04:00
Ryan Ernst 2c3ea43f45
HLRC: Add xpack usage api (#31975)
This commit adds the _xpack/usage api to the high level rest client.
Currently in the transport api, the usage data is exposed in a limited
fashion, at most giving one level of helper methods for the inner keys
of data, but then exposing thos subobjects as maps of objects. Rather
than making parsers for every set of usage data from each feature, this
PR exposes the entire set of usage data as a map of maps.
2018-07-13 09:33:27 -07:00
Christoph Büscher 69c2b2d802 [Tests] Mute failing SearchDocumentationIT#testSearchRequestAggregations 2018-07-13 10:56:47 +02:00
Martijn van Groningen ae5c70e603
muted tests due to #31940 2018-07-12 11:51:33 +02:00
James Baiera 5bcdff73d7
Add Snapshots Status API to High Level Rest Client (#31515)
This PR adds the Snapshots Status API to the Snapshot Client, as 
well as additional documentation for the status api.
2018-07-11 12:07:31 -04:00
Nik Everett eda6d182b5
Switch low level rest tests to new style Requests (#31938)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `client/rest` project to use the new versions.
2018-07-11 09:48:47 -04:00
Nik Everett aa6a1c5ca0
Switch high level rest tests to new style requests (#31937)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `client/rest-high-level` project to use the new
versions.
2018-07-11 09:18:04 -04:00
Nik Everett 4d83a0dd5a
HLREST: Bundle the x-pack protocol project (#31904)
The `:x-pack:protocol` project is an implementation detail shared by the
xpack projects and the high level rest client and really doesn't deserve
its own maven coordinants and published javadoc. This change bundles
`:x-pack:protocol` into the high level rest client.

Relates to #29827
2018-07-10 17:59:15 -04:00
Nik Everett dcbb1154bf
HLRest: Move xPackInfo() to xPack().info() (#31905)
Originally I put the X-Pack info object into the top level rest client
object. I did that because we thought we'd like to squash `xpack` from
the name of the X-Pack APIs now that it is part of the default
distribution. We still kind of want to do that, but at least for now we
feel like it is better to keep the high level rest client aligned with
the other language clients like C# and Python. This shifts the X-Pack
info API to align with its json spec file.

Relates to #31870
2018-07-10 13:01:28 -04:00