Commit Graph

6931 Commits

Author SHA1 Message Date
Costin Leau 353c0c500b Fix and enable datetime tests (elastic/x-pack-elasticsearch#2680)
Firstly, data in H2 is now stored in TIMESTAMP WITH TIME ZONE since H2
does not allow a global TZ to be set and picks the JVM TZ when a record
is read.
JdbcAssert is now aware of this allows TIMESTAMP with TZ == TIMESTAMP

Discovered a serious bug in DateTimeFunction - unfortunately date
histogram is not useful except for year since most extract functions
avoid ordering which a histogram preserves.
Thus most DTF are now terms aggs with scripting.

Improved a bug that caused duplicate functions to not be detected because
of aliasing.

Moved some datetime tests to CSV but the aggs tests now are in sync with
H2

Fixed bug that caused arithmetic on aggs to not be properly resolved by
splitting the processor definition tree to aggName (unresolved) and
aggPath (resolved)

Original commit: elastic/x-pack-elasticsearch@e75ada68f1
2017-10-13 13:52:48 +03:00
Tanguy Leroux a6776cef97 [Monitoring] Add interval_ms to Monitoring documents (elastic/x-pack-elasticsearch#2650)
This commit adds a new interval_ms field to the monitoring documents. 
This field indicates the current collection interval for Elasticsearch or 
external monitored systems. The value is indexed as a long.

Related to elastic/x-pack-elasticsearch#212

Original commit: elastic/x-pack-elasticsearch@2ceb20455c
2017-10-13 11:18:47 +02:00
David Roberts 10cc0088e4 [DOCS] Make clearer that xpack.xyz.enabled settings are node settings (elastic/x-pack-elasticsearch#2731)
The discussion in elastic/x-pack-elasticsearch#2697 shows that this was not clear before.

relates elastic/x-pack-elasticsearch#2697

Original commit: elastic/x-pack-elasticsearch@87553faa2c
2017-10-13 09:22:21 +01:00
Costin Leau 2bbb86eff7 Eliminate NOCOMMITS per discussion
Original commit: elastic/x-pack-elasticsearch@bd091a6608
2017-10-13 00:33:16 +03:00
Costin Leau 21c8a9168b Add antlr to qa tests for IDE execution
Original commit: elastic/x-pack-elasticsearch@d146b98be3
2017-10-12 23:59:17 +03:00
Nik Everett 69d1a5c5dd Give audit logs more time to show up in SQL tests
CI has been failing since we had to remove our tweaks to the audit
logging configuration due to
https://github.com/elastic/x-pack-elasticsearch/issues/2705

So we increase the timeout. The timeout is very very long, but it
seems like we need it for CI which is often slow.

Original commit: elastic/x-pack-elasticsearch@91a926a031
2017-10-12 13:53:09 -04:00
Nik Everett 69a326dd94 Fix CliErrorsIT in sql security tests
It was disabled because the CLI didn't work with security but
we've since fixed that so we can enable it.

Original commit: elastic/x-pack-elasticsearch@8d9b5ad89b
2017-10-12 12:42:13 -04:00
Nik Everett 852af7de57 Fix error handling in SQL's CLI (elastic/x-pack-elasticsearch#2730)
We weren't returning errors correctly from the server
or catching them correctly in the CLI. This fixes that
and adds simple integration tests.

Original commit: elastic/x-pack-elasticsearch@259da0da6f
2017-10-12 16:32:15 +00:00
Jay Modi 9028c0a642 Allow PkiRealm to use truststore.password setting (elastic/x-pack-elasticsearch#2727)
This change fixes an incorrect check for a missing password setting for the PKI realm. The check
only allowed the secure setting to be used for the PkiRealm password even though the legacy setting
is still valid. This change fixes the check.

Relates elastic/x-pack-elasticsearch#2487

Original commit: elastic/x-pack-elasticsearch@a4524c2c05
2017-10-12 10:07:08 -06:00
Nik Everett 6478713304 Add support for username and password in SQL CLI (elastic/x-pack-elasticsearch#2718)
Add support for username and password in SQL CLI and adds tests that CLI works with security.

Original commit: elastic/x-pack-elasticsearch@39c8dbfc97
2017-10-12 15:55:29 +00:00
lcawley 7f37c2c431 [DOCS] Fixed broken link to java transport client
Original commit: elastic/x-pack-elasticsearch@f3036bb2a0
2017-10-12 14:29:20 +01:00
Jason Tedor aece28c286 Add test for global checkpoint sync with security
After a write operation on an index, a post-operation global checkpoint
sync fires. Previously, this action fired on the same user as executed
the write action. If the user did not have priviledges for this action,
the global checkpoint sync would fail. With an upstream change in core,
this action now fires as the system user. This commit adds a test that
create a user that has minimal write permissions on an index, but none
that would imply it could execute the global checkpoint sync. This then
serves as a test that the upstream change to fire the global checkpoint
sync as the system user is correct. This test must run as a mulit-node
test so that a replica is a assigned so that the global checkpoint sync
fires in the first place. This test does indeed fail without the
upstream change, and passes with it.

Relates elastic/x-pack-elasticsearch#2744

Original commit: elastic/x-pack-elasticsearch@bf7e771756
2017-10-12 09:19:17 -04:00
lcawley b628815dbe [DOCS] Fixed link to X-Pack transport client
Original commit: elastic/x-pack-elasticsearch@0870334e4b
2017-10-12 13:41:14 +01:00
Lisa Cawley 604229cd4d [DOCS] Added transport client info for X-Pack (elastic/x-pack-elasticsearch#2737)
* [DOCS] Added transport client info for X-Pack

* [DOCS] Relocated X-Pack java client info

* [DOCS] Added transport client deprecation info

Original commit: elastic/x-pack-elasticsearch@416aab1d76
2017-10-12 13:18:44 +01:00
Costin Leau fa4504ed28 Fix some NOCOMMITs
Original commit: elastic/x-pack-elasticsearch@1a6ac1e6c6
2017-10-12 14:24:56 +03:00
Costin Leau e3d072aeea Make JDBC driver throw only SQLException (elastic/x-pack-elasticsearch#2708)
A JDBC driver should throw only checked SQLExceptions.
Introduce JdbcSQLException and fix some no-commits along the way.

Original commit: elastic/x-pack-elasticsearch@299fcf9ace
2017-10-12 12:27:19 +03:00
Tanguy Leroux 0299886388 [Tests] Use XPack Usage API to verify Monitoring exporters are disabled (elastic/x-pack-elasticsearch#2648)
This commit changes the MonitoringIt and XPackRestIT tests so that the
disableMonitoring() method now use the XPack Usage API in order to check
that the monitoring exporters are correctly disabled. It checks at the
beginning of the tests (all exporters must be disabled before running
the test) and also at the end of the test.

This commit also fixes a bug in MonitoringIT where the Bulk thread pool
active queue was wrongly extracted from the response's map, forcing the
test to always wait for 30sec.

relates elastic/x-pack-elasticsearch#2459

Original commit: elastic/x-pack-elasticsearch@2d349e840f
2017-10-12 09:36:44 +02:00
Igor Motov 7f0c44f138 SQL: Fix string to boolean conversion (elastic/x-pack-elasticsearch#2728)
This commit also adds tests for conversion to boolean

Original commit: elastic/x-pack-elasticsearch@d7740929a2
2017-10-11 19:56:36 -04:00
Igor Motov 06b4c043b9 SQL: Add toXcontent test for SqlResponse (elastic/x-pack-elasticsearch#2726)
Original commit: elastic/x-pack-elasticsearch@aa7b8f1a91
2017-10-11 19:55:59 -04:00
Costin Leau 2026198dd4 Add size for column tests (elastic/x-pack-elasticsearch#2685)
Add displaySize to columnInfo

Original commit: elastic/x-pack-elasticsearch@ed1d265e98
2017-10-12 00:03:41 +03:00
Costin Leau c9a41e111e Improve Sql exception reporting (elastic/x-pack-elasticsearch#2679)
Better handling of SQL exceptions (result of incorrect queries) vs
unexpected ones (engine failure, ES...)

Original commit: elastic/x-pack-elasticsearch@2698402cdb
2017-10-11 23:23:52 +03:00
Costin Leau 57fcbb81cb SQL: Improve JDBC communication (elastic/x-pack-elasticsearch#2660)
* Improve JDBC communication
Jdbc HTTP client uses only one url for messages and relies on / for ping
Fixed ES prefix being discarded (missing /)
Add HEAD handler for JDBC endpoint

Original commit: elastic/x-pack-elasticsearch@389f82262e
2017-10-11 23:03:03 +03:00
Costin Leau 6a1806a3eb Fix line too long
Original commit: elastic/x-pack-elasticsearch@084095a944
2017-10-11 22:32:14 +03:00
Costin Leau d6881e25c9 Use embedded client inside the planner as well
Original commit: elastic/x-pack-elasticsearch@c9d20672be
2017-10-11 22:08:20 +03:00
Nik Everett 125f140620 Fix SQL CLI tests in windows (elastic/x-pack-elasticsearch#2738)
We put the CLI in unix mode so if we send it
`\r\n` (the default in windows) then it'll
spit out extra "you are on a line continuation"
characters (`|`). Instead, we can use `\n`
directly and everything works.

I've also added a timeout to the reads from the
CLI because it makes the tests easier to debug.

Original commit: elastic/x-pack-elasticsearch@69f69f4092
2017-10-11 18:31:45 +00:00
Tanguy Leroux ea4bff1d43 [Monitoring] Align MonitoringBulkDoc serialization with 6.0 (elastic/x-pack-elasticsearch#2736)
The version used in serialization must be aligned with 6.0/6.x.

Original commit: elastic/x-pack-elasticsearch@db63b91bc6
2017-10-11 17:56:24 +02:00
Lisa Cawley 2455415a04 [DOCS] Small fixes in the overall buckets API (elastic/x-pack-elasticsearch#2732)
* [DOCS] Small fixes in the overall buckets API

* [DOCS] Addressed feedback in overall buckets API

Original commit: elastic/x-pack-elasticsearch@4f79bc9a50
2017-10-11 16:25:05 +01:00
lcawley 723dd49905 [DOCS] Fixed typo in count function
Original commit: elastic/x-pack-elasticsearch@34c821796b
2017-10-11 16:16:28 +01:00
Tanguy Leroux 8484680007 Few fixes in packaging tests
This commit fixes indentation in certgen.bash, adds a check on cluster
health in bootstrap_password.bash and fixes a bug in xpack.bash

Original commit: elastic/x-pack-elasticsearch@d6847f6640
2017-10-11 11:53:10 +02:00
David Roberts c84d69fde3 [DOCS] Fix ML post_data docs (elastic/x-pack-elasticsearch#2689)
It was pointed out in
https://github.com/elastic/elasticsearch-net/pull/2856#discussion_r142830656
that our post_data docs incorrectly say that reset_start and reset_end are
body parameters.  In fact they are query parameters.

There were also a number of other errors and ommissions on this page that I
have attempted to correct.

Original commit: elastic/x-pack-elasticsearch@c83decacc7
2017-10-11 10:47:07 +01:00
Tim Vernum a4f7db4f66 [Security] Improve error messages in setup-passwords (elastic/x-pack-elasticsearch#2724)
Provides more verbose messaging around errors and possible causes when the tool aborts.

This change is primarily focused on errors connecting to the Elasticsearch node when TLS is enabled on the HTTP connection.

Original commit: elastic/x-pack-elasticsearch@aa8f7c6143
2017-10-11 12:32:35 +10:00
Tim Vernum bc038b323d [Security] Apply validation when parsing certgen input (elastic/x-pack-elasticsearch#2711)
When certgen configuration was read from an input file (`-in` option) validation errors were collected but never reported. Depending on the type of error this may have caused the tool to exit with an internal error (e.g. NPE).

Validation is now applied after parsing the file and if errors are found the tool exits.

Original commit: elastic/x-pack-elasticsearch@b2262ed1d7
2017-10-11 12:30:19 +10:00
Nik Everett b02e569b38 Speed up audit log testing for SQL (elastic/x-pack-elasticsearch#2721)
SQL was cleaning up the audit logs *after* each test
but this switches it to cleaning up the audit logs
*before* each test. This is faster because we can
generate a lot of audit logs before the first test.

Original commit: elastic/x-pack-elasticsearch@71d8f76667
2017-10-10 16:48:37 -04:00
Nik Everett 9633b1d7bc Remove an @AwaitsFix on test in SQL (elastic/x-pack-elasticsearch#2719)
It has been fixed already, actually.

Original commit: elastic/x-pack-elasticsearch@46c8dacc50
2017-10-10 16:44:21 -04:00
Costin Leau 1cd6fb23ec Make SearchCursor limit aware
Original commit: elastic/x-pack-elasticsearch@c4839bc293
2017-10-10 19:22:42 +03:00
David Roberts 5d0388ccb3 [TEST] Fix ML node attribute test
When ML is disabled the attribute checking is stricter, but the test
did not reflect this

Original commit: elastic/x-pack-elasticsearch@613e97c595
2017-10-10 16:22:03 +01:00
David Roberts ab9cc25a8e [ML] Prevent ML node attributes being set directly (elastic/x-pack-elasticsearch#2725)
ML uses node attributes to ensure that the master node knows how many
ML jobs may be allocated to each node.  This change prevents a user
messing up the way these attributes are used by setting them differently
using node.attr.* entries in their elasticsearch.yml.

This covers the "very short term" change outlined in elastic/x-pack-elasticsearch#2649

Original commit: elastic/x-pack-elasticsearch@9c381801d9
2017-10-10 15:12:59 +01:00
Dimitris Athanasiou 5eea355b33 [ML] Add overall buckets api (elastic/x-pack-elasticsearch#2713)
Adds the GET overall_buckets API.

The REST end point is: GET
/_xpack/ml/anomaly_detectors/job_id/results/overall_buckets

The API returns overall bucket results. An overall bucket
is a summarized bucket result over multiple jobs.
It has the `bucket_span` of the longest job's `bucket_span`.
It also has an `overall_score` that is the `top_n` average of the
max anomaly scores per job.

relates elastic/x-pack-elasticsearch#2693

Original commit: elastic/x-pack-elasticsearch@ba6061482d
2017-10-10 14:41:24 +01:00
Dimitris Athanasiou 90e327032e [ML][DOCS] Fix bucket_span type in results resources (elastic/x-pack-elasticsearch#2714)
Original commit: elastic/x-pack-elasticsearch@af24bde71f
2017-10-10 12:51:12 +01:00
Tanguy Leroux 1ed4be1471 Show exit code in Bootstrap Password packaging tests
Also cleans up some files before the test is executed,
and explicitly binds to 127.0.0.1/9200.

Original commit: elastic/x-pack-elasticsearch@778584ea78
2017-10-10 09:51:22 +02:00
Alexander Reelsen 80593fb23c Watcher: Add execution state to watch status (elastic/x-pack-elasticsearch#2699)
The execution state is kind of a global indicator if a watch has been
running successfully and is used by the watcher UI.

However this field is only stored in the watch history but not part of
the watch status, thus it is not available everywhere. In order to
simplify the watcher UI this commit also adds the field to the
watch status which is stored together with the watch.

It is stored under the `status.execution_state` field as `status.state`
is already taken. This is also reflects with the name of the java class.

The WatchStatus class does not contain serialization checks, as this is
intended to be backported to 6.x, where those checks will be added.

Once the backport is done, the old execution state field can be fully
deleted from the master branch in another commit (syncing with Kibana
folks required).

relates elastic/x-pack-elasticsearch#2385

* fix doc tests

Original commit: elastic/x-pack-elasticsearch@26e8f99571
2017-10-10 09:07:33 +02:00
Alexander Reelsen cadfd03529 Watcher: Allow JIRA path to be custom chosen (elastic/x-pack-elasticsearch#2682)
The path of a JIRA endpoint used to be fixed. This commit allows the
path to be dynamic, so that users can deploy their JIRA instance under
an arbitrary prefix.

Original commit: elastic/x-pack-elasticsearch@7702505114
2017-10-10 08:55:28 +02:00
Igor Motov ed712d0e3f SQL: clean unused Exceptions and replace Util functions with core versions (elastic/x-pack-elasticsearch#2654)
This commit removes ThrowableConsumer, WrappingException, ActionUtils and ObjectUtils by replacing them with core equivalents when needed.

Original commit: elastic/x-pack-elasticsearch@5a68418a3d
2017-10-09 18:00:29 -04:00
Chris Earle 69ab7797be [Monitoring] Cleaner Service should be able to cleanup .watcher-history* (elastic/x-pack-elasticsearch#2696)
This adds a dynamic setting, which defaults to `false` currently, that can be used to delete all `.watcher-history*` indices that match the same age requirements as Monitoring indices.

Original commit: elastic/x-pack-elasticsearch@8ca3bdbca3
2017-10-09 15:46:07 -06:00
Igor Motov fa410095ce SQL: remove synchronous execute method from Command (elastic/x-pack-elasticsearch#2653)
We switched most of the commands to asynchronous mode, this commit removes the synchronous version of exec to prevent accidental use of it in the future.

Original commit: elastic/x-pack-elasticsearch@03d0a6350d
2017-10-09 16:01:56 -04:00
Igor Motov 79d6b88763 Merge remote-tracking branch 'elastic/master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@7503023447
2017-10-09 15:47:35 -04:00
David Roberts 9ad961088d [TEST] Wait a little longer for named pipes to open in unit tests (elastic/x-pack-elasticsearch#2712)
Same fix as elastic/x-pack-elasticsearch#987, but for the unit tests.  The slowness affecting EBS
volumes created from snapshots can affect CI as it runs on AWS instances.

Original commit: elastic/x-pack-elasticsearch@306b8110b7
2017-10-09 13:09:17 +01:00
Albert Zaharovits 98347088f9 Fix LDAP Authc connections deadlock (elastic/x-pack-elasticsearch#2587)
Do not execute bind on on the LDAP reader thread

Each LDAP connection has a single associated thread, executing the handlers for async requests; this is managed by the LDAP library. The bind operation is blocking for the connection. It is a deadlock to call bind, if on the LDAP reader thread for the same connection, because waiting for the bind response blocks the thread processing responses (for this connection).
This will execute the bind operation (and the subsequent runnable) on a thread pool after checking for the conflict above.

Closes: elastic/x-pack-elasticsearch#2570, elastic/x-pack-elasticsearch#2620

Original commit: elastic/x-pack-elasticsearch@404a3d8737
2017-10-09 13:06:12 +03:00
Yannick Welsch 20c0e01523 Set minimum_master_nodes on rolling-upgrade test
Companion commit to elastic/elasticsearch#26911

Original commit: elastic/x-pack-elasticsearch@dcdbd14f78
2017-10-09 10:59:58 +02:00
Simon Willnauer cd14f33ae2 Return List instead of an array from settings (elastic/x-pack-elasticsearch#2694)
XPack side of elastic/elasticsearch#26903

Original commit: elastic/x-pack-elasticsearch@f0390974ab
2017-10-09 09:52:34 +02:00