Commit Graph

6105 Commits

Author SHA1 Message Date
Nik Everett edcc87e30e `gradle check -xforbiddenPatterns` passes!
Hurray! Now we just have to keep it that way!

Original commit: elastic/x-pack-elasticsearch@98b8ede44e
2017-07-06 16:24:19 -04:00
Nik Everett fc0bce0a3e Some more tests
Closer to `gradle check -xforbiddenPatterns` passing.

Original commit: elastic/x-pack-elasticsearch@fbce74a1f3
2017-07-06 15:58:12 -04:00
Nik Everett 527541a66e More build fixes
* A unit test for cli
* Licenses for cli
* Remove licenses for protos (no more deps)
* `SHOW TABLES` returns results in order (makes testing easier)
* Clean up embedded jdbc server
* Wire up embedded cli server

Original commit: elastic/x-pack-elasticsearch@b98aaf446b
2017-07-06 14:37:09 -04:00
Nik Everett d25c23c3b4 Fix some forbiddens in cli
Original commit: elastic/x-pack-elasticsearch@3dc96c7279
2017-07-06 10:54:41 -04:00
Nik Everett bc2f0fe7ff Get jdbc tests passing
`gradle check -xforbiddenPatterns` now passes in jdbc.

This makes running the embedded HTTP server slightly more difficult,
you now have to add the following to your jvm arguments.
```
-ea -Dtests.rest.cluster=localhost:9200 -Dtests.embed.sql=true -Dtests.security.manager=false
```

Depending on your environment the embedded jdbc connection may give
spurious failures that look like:
```
org.elasticsearch.xpack.sql.jdbc.jdbc.JdbcException: RemoteTransportException[[node-0][127.0.0.1:9300][indices:data/read/search]]; nested: SearchPhaseExecutionException[]; nested: GeneralScriptException[Failed to compile inline script [( params.a0 > params.v0 ) && ( params.a1 > params.v1 )] using lang [painless]]; nested: CircuitBreakingException[[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_per_minute] setting];
...
Caused by: Failed to execute phase [fetch],
..
Caused by: GeneralScriptException[Failed to compile inline script [( params.a0 > params.v0 ) && ( params.a1 > params.v1 )] using lang [painless]]; nested: CircuitBreakingException[[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_per_minute] setting];
...
Caused by: CircuitBreakingException[[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_per_minute] setting]
```

`gradle check` works around this by setting `script.max_compilations_per_minute`
to `1000`.

Another change is that we no longer support loading the test data by
uncommenting some code. Instead we load the test data into Elaticsearch
before the first test and we deleted it after the last test. This is
so that tests that required different test data can interoperate with
eachother. The spec tests all use the same test data but the metadata
tests do not.

Original commit: elastic/x-pack-elasticsearch@8b8f684ac1
2017-07-06 10:24:59 -04:00
Nik Everett 18bc094dbc Remove LocalEsCluster
We aren't using it anymore.

Original commit: elastic/x-pack-elasticsearch@67f47c4f11
2017-07-05 08:21:35 -04:00
Nik Everett 9d83eccf27 Merge branch 'feature/sql' of github.com:elastic/x-pack-elasticsearch into feature/sql
Original commit: elastic/x-pack-elasticsearch@34d536c5e1
2017-07-05 08:15:11 -04:00
Nik Everett b2484bebe8 Merge branch 'master' into feature/sql
This should get us OK until 0170e0e8d3.

Original commit: elastic/x-pack-elasticsearch@e0bea08365
2017-07-05 08:14:24 -04:00
Dimitrios Athanasiou 3057f7f4b6 [TEST] Fix post data request in post_data.yml
Original commit: elastic/x-pack-elasticsearch@14e9083c02
2017-07-05 12:06:53 +01:00
Dimitris Athanasiou 15f9b1ed9c [ML] Impove mechanism for ignoring maintenance windows (elastic/x-pack-elasticsearch#1914)
Currently, the autodetect process has an `ignoreDowntime`
parameter which, when set to true, results to time being
skipped over to the end of the bucket of the first data
point received. After that, skipping time requires closing
and opening the job. With regard to datafeeds, this does not
work well with real-time requests which use the advance-time
API in order to ensure results are created for data gaps.

This commit improves this functionality by making it more
flexible and less ambiguous.

- flush API now supports skip_time parameter which
sends a control message to the autodetect process
telling it to skip time to a given value
- the flush API now also returns the last_finalized_bucket_end
time which allows clients to resume data searches correctly
- the datafeed start API issues a skip_time request when the
given start time is after the resume point. It then resumes
the search from the last_finalized_bucket_end time.

relates elastic/x-pack-elasticsearch#1913


Original commit: elastic/x-pack-elasticsearch@caa5fe8016
2017-07-05 11:33:42 +01:00
Lisa Cawley 595468a505 [DOCS] Update doc links for watcher APIs (elastic/x-pack-elasticsearch#1880)
Original commit: elastic/x-pack-elasticsearch@47521f1261
2017-07-04 11:16:47 -07:00
Lisa Cawley 04ff94a180 [DOCS] Update doc links for ML APIs (elastic/x-pack-elasticsearch#1882)
Original commit: elastic/x-pack-elasticsearch@cdc45f282d
2017-07-04 11:10:34 -07:00
Costin Leau 12e12f7a94 Improve exception reporting
Add the source spec in the error stacktrace

Original commit: elastic/x-pack-elasticsearch@bc7ed6ac60
2017-07-04 19:25:03 +03:00
Costin Leau 6f670c8d8e Testing fixes
- simplify handling of timezone in H2
- fix leaking threadpool in HttpServer
- update Csv tests
- keep the dates as long in internal Page

Original commit: elastic/x-pack-elasticsearch@43a804607f
2017-07-04 19:05:09 +03:00
Sophie Chang 0ff26e7566 [DOCS] Added limitation for time-based index patterns (elastic/x-pack-elasticsearch#1910)
* [DOCS] Added limitation for time-based index patterns

* [DOCS] Updated limitations

Original commit: elastic/x-pack-elasticsearch@9927105cc0
2017-07-04 08:40:42 -07:00
David Roberts b93890b0df [ML] Add license downgrade warning (elastic/x-pack-elasticsearch#1912)
When applying a license that will cause ML to be disabled, a warning
to this effect is now included in the acknowledgement messages.

relates elastic/x-pack-elasticsearch#1888

Original commit: elastic/x-pack-elasticsearch@e453f20f58
2017-07-04 15:10:18 +01:00
Dimitrios Athanasiou 05a73a32bd [TEST] Correctly verify that aliases were deleted in MlJobIT
This fixes `testDeleteJobAfterMissingAliases` to not fail randomly.
The reason the test was failing is that at some point some aliases
are deleted and the cat-aliases API is called to verify they were
indeed deleted. This was checked by asserting an
index_not_found_exception was thrown by the cat-aliases request.
This was some times working as there were no other aliases. However,
that depends on whether other x-pack features had time to create their
infrastructure. For example, security creates an alias. When other
aliases had the time to be created, the cat-aliases request does not
fail and the test fails.

This commit simply changes the verification that the read/write
aliases were deleted by replacing the cat-aliases request with
two single get-alias requests.

Original commit: elastic/x-pack-elasticsearch@fe2c7b0cb4
2017-07-04 13:14:38 +01:00
Alexander Reelsen fff72256a5 Watcher: Fix wrong logging in reporting attachment parser (elastic/x-pack-elasticsearch#1900)
The logging shows a wrong HTTP response status code from a previous
request. In addition the body now also gets logged, as debugging
is impossible otherwise.

Original commit: elastic/x-pack-elasticsearch@cc998cd587
2017-07-04 13:01:14 +02:00
Nik Everett 50a46d0ed2 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@51fc29da6a
2017-07-03 18:04:09 -04:00
Costin Leau 83d73c9b3a More polishing
Original commit: elastic/x-pack-elasticsearch@043edf250b
2017-07-04 00:41:16 +03:00
Costin Leau 405b4a4166 Update test dataset name
Original commit: elastic/x-pack-elasticsearch@49951836c1
2017-07-04 00:00:11 +03:00
Nik Everett 56803bdd1b Port some of CLI's demos to tests
Original commit: elastic/x-pack-elasticsearch@5827c08268
2017-07-03 16:41:25 -04:00
Costin Leau 3eb1258b0d Add to CSV infra plus some refactoring of the JDBC one
Original commit: elastic/x-pack-elasticsearch@3e002c91c2
2017-07-03 21:01:29 +03:00
Clinton Gormley fd518ea020 Include shared/attributes.asciidoc directly from docs master
Original commit: elastic/x-pack-elasticsearch@f3a0828c5d
2017-07-03 18:23:19 +02:00
Christoph Büscher 8d26996afd Remove QueryParseContext (elastic/x-pack-elasticsearch#1895)
This is the x-pack side of elastic/elasticsearch#25486.

Original commit: elastic/x-pack-elasticsearch@c90a3e096b
2017-07-03 17:31:18 +02:00
Simon Willnauer 20f6d66294 Adopt to network settings cleanup in elastic/elasticsearch#25489
Original commit: elastic/x-pack-elasticsearch@364bb260ee
2017-07-02 10:17:30 +02:00
Costin Leau bc9c9a9151 Expanding the JDBC testing framework
Original commit: elastic/x-pack-elasticsearch@cc31f7b6a9
2017-07-01 17:20:48 +03:00
Costin Leau 16ea84defe Fix some checkstyles
Original commit: elastic/x-pack-elasticsearch@ff1c6a6e70
2017-07-01 17:19:46 +03:00
Chris Earle 02c0ad2aad [Monitoring] Reduce NodeStats Collection to required Data (elastic/x-pack-elasticsearch#1240)
This changes from collecting every index statistic to only what we actually want. This should help to reduce the performance impact of the lookup.

Original commit: elastic/x-pack-elasticsearch@80ae20f382
2017-06-30 19:42:44 -04:00
Nik Everett 4bb605d819 Fix licenses in server
Original commit: elastic/x-pack-elasticsearch@a3207814f8
2017-06-30 19:39:06 -04:00
Nik Everett 1a2cbea747 Checkstyle
Original commit: elastic/x-pack-elasticsearch@4d8e9b2a7a
2017-06-30 17:03:50 -04:00
Nik Everett 6de905c2e8 Enable more tests
And remove extra file

Original commit: elastic/x-pack-elasticsearch@aade36e030
2017-06-30 16:43:37 -04:00
Nik Everett 336980ec28 Add time zone support to the connection
Flows time zones through the `QueryInitRequest` and into the
`ExpressionBuilder` which attaches the time zones to date/time
expressions. Modifies the code that generates date aggs,
scripts, and extracts results to use the time zones.

Original commit: elastic/x-pack-elasticsearch@d6682580d1
2017-06-30 16:43:37 -04:00
Nik Everett 514b7290d7 Fix checkstyle
Original commit: elastic/x-pack-elasticsearch@f9b2d6ee47
2017-06-30 16:39:31 -04:00
Tim Brooks 7b3b2d5f02 Localhost check: check if addr bound to interface (elastic/x-pack-elasticsearch#1901)
This is related to elastic/x-pack-elasticsearch#1217 and elastic/x-pack-elasticsearch#1896. Right now we are checking if an
incoming address is the loopback address or a special local addres. It
appears that we also need to check if that address is bound to a
network interface to be thorough in our localhost check.

This change mimicks how we check if localhost in `PatternRule`.

Original commit: elastic/x-pack-elasticsearch@a8947d6174
2017-06-30 14:19:49 -05:00
Costin Leau be2153851c Synch more functionality
Original commit: elastic/x-pack-elasticsearch@093c275b85
2017-06-30 20:41:47 +03:00
Dimitrios Athanasiou 8264cbf72f [TEST] Stabilise UpdateInterimResultsIT
Depending on the random numbers fed to the analytics,
it is possible that the first planted anomaly ends up
in a different bucket due to the overlapping buckets feature.
Then that may result to a single interim bucket being available
due to overlapping buckets blocking the other interim bucket
from being considered.

I am removing the initial anomaly from the test as it is not useful
and it makes the test unstable.

relates elastic/x-pack-elasticsearch#1897

Original commit: elastic/x-pack-elasticsearch@aca7870708
2017-06-30 17:11:54 +01:00
Christoph Büscher e7e24c453c Reenable SecurityIndexSearcherWrapperIntegrationTests (elastic/x-pack-elasticsearch#1894)
Original commit: elastic/x-pack-elasticsearch@03ff1bf9a5
2017-06-30 16:52:47 +02:00
David Roberts 10c37f0fa4 [TEST] Improve diagnostics for ML interim results test failure
Original commit: elastic/x-pack-elasticsearch@2ccc9d71ae
2017-06-30 11:47:53 +01:00
Tim Brooks 76bf3ba767 Bring back disabling-default-password docs section
There are multiple references to this section in different areas of the
documentation. This commit brings back this section to fix the build.

A more extensive PR updating the documentation for "no default
password" work will follow up.

Original commit: elastic/x-pack-elasticsearch@0378e78c8a
2017-06-29 16:23:58 -05:00
Tim Brooks c773115c39 Remove doc reference disabling-default-password
This section has been removed from setting-up-authentication. This
commit removes a reference to this section that no longer exists.

Original commit: elastic/x-pack-elasticsearch@43aa0077f9
2017-06-29 16:01:44 -05:00
Jay Modi a9707a461d Use a secure setting for the watcher encryption key (elastic/x-pack-elasticsearch#1831)
This commit removes the system key from master and changes watcher to use a secure setting instead
for the encryption key.

Original commit: elastic/x-pack-elasticsearch@5ac95c60ef
2017-06-29 14:58:35 -06:00
Tim Brooks f2cbe20ea0 Remove default passwords from reserved users (elastic/x-pack-elasticsearch#1665)
This is related to elastic/x-pack-elasticsearch#1217. This PR removes the default password of
"changeme" from the reserved users.

This PR adds special behavior for authenticating the reserved users. No
ReservedRealm user can be authenticated until its password is set. The
one exception to this is the elastic user. The elastic user can be
authenticated with an empty password if the action is a rest request
originating from localhost. In this scenario where an elastic user is
authenticated with a default password, it will have metadata indicating
that it is in setup mode. An elastic user in setup mode is only
authorized to execute a change password request.

Original commit: elastic/x-pack-elasticsearch@e1e101a237
2017-06-29 15:27:57 -05:00
Christoph Büscher 075eda4fc1 Temporarily disable SecurityIndexSearcherWrapperIntegrationTests
Original commit: elastic/x-pack-elasticsearch@bcef6ae8c6
2017-06-29 20:46:30 +02:00
Nik Everett 0e8ef06947 Move sql's server stuff into :x-pack-elasticsearch:sql:server
This should make it easier to run stuff for just sql.

Original commit: elastic/x-pack-elasticsearch@d68465edb0
2017-06-29 14:31:23 -04:00
Nik Everett 2f9c0858f7 More build fixes
Original commit: elastic/x-pack-elasticsearch@8b241ea4d1
2017-06-29 12:59:56 -04:00
Nik Everett e5df9a0269 Fix remaining warnings in jdbc and drop jdk exemptions
Original commit: elastic/x-pack-elasticsearch@aff8a7fe42
2017-06-29 12:16:50 -04:00
Nik Everett acfc8d1bc5 Add hack to run internal test cluster if needed
Runs an internal test cluster if the test is not started with
an external rest test cluster.

Original commit: elastic/x-pack-elasticsearch@ce3d6f64e2
2017-06-29 12:07:39 -04:00
Christoph Büscher 7c6b8ffa36 Adapting to changes in https://github.com/elastic/elasticsearch/pull/25448 (elastic/x-pack-elasticsearch#1887)
Original commit: elastic/x-pack-elasticsearch@5cdf5a2372
2017-06-29 17:10:34 +02:00
Nik Everett 9858d2ea3c Migrate remaining compare tests to IT
Migrates the remaining comparison tests to integration tests
so they run easilly in gradle against real Elasticsearch nodes.

This breaks running them in the IDE without running an Elasticsearch
node and setting `-Drest.test.cluster=localhost:9200`. You can run
a compatible node with `cd sql-clients/jdbc && gradle run`.

Original commit: elastic/x-pack-elasticsearch@90a7c2dae7
2017-06-29 10:15:49 -04:00