Commit Graph

6812 Commits

Author SHA1 Message Date
Costin Leau a7b559c825 temporarily set UTC as timezone to comply with H2 behavior
Original commit: elastic/x-pack-elasticsearch@00be17cebb
2017-10-28 11:59:13 +03:00
Nik Everett f92684c436 Fix SQL test
Original commit: elastic/x-pack-elasticsearch@14d07c2607
2017-10-27 17:38:34 -04:00
Nik Everett b91e9d85e9 Fix SHAs
Original commit: elastic/x-pack-elasticsearch@15c2669416
2017-10-27 17:12:15 -04:00
Costin Leau f26a9acdc4 Update ANTLR in qa project as well
Original commit: elastic/x-pack-elasticsearch@58c10398b4
2017-10-27 23:43:37 +03:00
Costin Leau 660cb3b814 Upgrade ANTLR to 4.5.3
Relates to elastic/x-pack-elasticsearch#26834

Original commit: elastic/x-pack-elasticsearch@57ba8f1d5f
2017-10-27 23:29:00 +03:00
Costin Leau 8d5572a77b SQL: improve validation of properties (both names and values) (elastic/x-pack-elasticsearch#2759)
* refactor Configuration class

move away from Properties
perform validation of settings names and values at startup
better exception handling

Original commit: elastic/x-pack-elasticsearch@d8a9edeccf
2017-10-27 22:54:03 +03:00
Costin Leau af591b9edd SQL: Remove usage of Settings inside SqlSettings (elastic/x-pack-elasticsearch#2757)
* Remove usage of Settings inside SqlSettings

Also hook client timeouts to the backend
Set UTC as default timezone when using CSV
As the JVM timezone changes, make sure to pin it to UTC since this is what the results are computed against

Original commit: elastic/x-pack-elasticsearch@3e7aad8c1f
2017-10-27 18:55:59 +03:00
Costin Leau 6b1d0d1c8e SQL: Remove time threshold for rule executor (elastic/x-pack-elasticsearch#2750)
Refactor RuleExecutor to eliminate time-based thresholds

Original commit: elastic/x-pack-elasticsearch@ba131f8058
2017-10-27 18:43:13 +03:00
Nik Everett 3d0f57d976 Add remaining security tests (elastic/x-pack-elasticsearch#2797)
This adds all of the security tests I think SQL is going to need for the initial release. SQL is still missing an entire scenario though: SSL enabled. Either way, this removes some `NOCOMMIT`s in `qa/sql/security`. Adding the SSL testing can come later.

Original commit: elastic/x-pack-elasticsearch@851620b606
2017-10-26 17:23:35 +00:00
Nik Everett 52d9de1de7 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@dd027d9ca5
2017-10-26 09:51:04 -04:00
lcawley 615de68711 [DOCS] Added xpack.watcher.history.cleaner_service_enabled
Original commit: elastic/x-pack-elasticsearch@ceead2a1c2
2017-10-25 16:35:20 -07:00
Lisa Cawley c67ec73cb5 [DOCS] Add xpack.ml.max_model_memory_limit (elastic/x-pack-elasticsearch#2787)
* [DOCS] Add xpack.ml.max.model_memory_limit

* [DOCS] Addressed feedback on model limit setting

Original commit: elastic/x-pack-elasticsearch@77a10bfe0e
2017-10-25 09:15:19 -07:00
Lee Hinman cdaa047d56 [TEST] Fix compilation for ARS stats exposing
Original commit: elastic/x-pack-elasticsearch@eb0ad99434
2017-10-24 11:09:19 -06:00
Lisa Cawley 73e819b0bd [DOCS] Add secure versions of SSL passphrases (elastic/x-pack-elasticsearch#2478)
* [DOCS] Add secure versions of SSL passphrases

* [DOCS] Add secure xpack settings

* [DOCS] Clarify type of keystore

* [DOCS] Added secure settings to security page

* [DOCS] Clarify X-Pack secure settings

* [DOCS] Reformat secure X-Pack settings

Original commit: elastic/x-pack-elasticsearch@efe043fb67
2017-10-24 16:38:37 +01:00
Lisa Cawley 3a6870f0dc [DOCS] Added SSL settings to X-Pack installation (elastic/x-pack-elasticsearch#2733)
Original commit: elastic/x-pack-elasticsearch@c88cce763a
2017-10-24 08:34:07 -07:00
Albert Zaharovits 403912b8a2 SecureSettings ignored by customAuditIndexSettings (elastic/x-pack-elasticsearch#2748)
customAuditIndexSettings does not submit SecureSettings with putIndexMapping.

relates elastic/x-pack-elasticsearch#2705

* Randomize SecureSetting in testcase

* Apply feedback

Original commit: elastic/x-pack-elasticsearch@1a5414b057
2017-10-24 13:50:35 +03:00
Alexander Reelsen 6f437c973a Watcher: Ensure all templates exist before starting watcher (elastic/x-pack-elasticsearch#2765)
This is to ensure that the required templates exist (which are added by
the WatcherIndexTemplateRegistry) before actually starting watcher.

Relates elastic/x-pack-elasticsearch#2761 

Original commit: elastic/x-pack-elasticsearch@568088061f
2017-10-23 11:57:40 +02:00
Martijn van Groningen c9682d02d4 fix test
Original commit: elastic/x-pack-elasticsearch@7ca5e0fb54
2017-10-23 09:58:33 +02:00
Martijn van Groningen 652f6560b8 security: Always allow access to a rootdoc's nested documents if access to rootdoc is allowed
relates elastic/x-pack-elasticsearch#2665

Original commit: elastic/x-pack-elasticsearch@2bbddd1dd2
2017-10-23 09:28:53 +02:00
Igor Motov 387944d176 SQL: Move special joda time handling into DocValueExtractor (elastic/x-pack-elasticsearch#2758)
Moves joda time handling into DocValueExtractor, that's the only place where it occurs at the moment.

Original commit: elastic/x-pack-elasticsearch@205e82990a
2017-10-22 15:54:00 +02:00
Nik Everett d7ab14ee54 Fix SQL security build's run config
Now it is the same as the integTest config agian. I should have done
this in elastic/x-pack-elasticsearch#2753 but I forgot.

Original commit: elastic/x-pack-elasticsearch@bbbb8b1dc7
2017-10-19 13:14:30 -04:00
Nik Everett 56ce29c6bf Security tests for SQL's CLI and JDBC (elastic/x-pack-elasticsearch#2770)
Add security tests for SQL's CLI and JDBC features. I do this by factoring out all the "actions" from the existing REST tests into an interface and implement it for REST, CLI, and JDBC. This way we can share the same audit log assertions across tests and we can be sure that the REST, CLI, and JDBC tests cover all the same use cases.

Original commit: elastic/x-pack-elasticsearch@82ff66a520
2017-10-19 17:13:31 +00:00
Alexander Reelsen b76c85e7fd Docs: Change port to be an integer in htttp input docs
Original commit: elastic/x-pack-elasticsearch@768ec54e03
2017-10-19 13:53:24 +02:00
Nik Everett 65f2b9fe01 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@9fc67cbfee
2017-10-18 10:27:19 -04:00
Simon Willnauer 2d1ce76194 Adopt core that `_flush` and `_force_merge` doesn't refresh anymore (elastic/x-pack-elasticsearch#2752)
Relates to elastic/elasticsearch#27000

Original commit: elastic/x-pack-elasticsearch@52e9951094
2017-10-16 10:16:50 +02:00
Nik Everett 770bc9516c Switch sql audit tests from index to the log file (elastic/x-pack-elasticsearch#2753)
This is *way* faster because we don't have to wait for the audit
events from previous test runs to drain into the index. And we
don't have to wait for the index's refresh cycle. We have to parse
the log lines which is a bit more brittle but it feels worth it
at this point.

Original commit: elastic/x-pack-elasticsearch@4b1758fc32
2017-10-14 12:27:51 +00:00
Nik Everett 269b656cf2 Prune the forbiddenPatterns tasks that are skipped (elastic/x-pack-elasticsearch#2754)
We don't need to blindly skip all of these tasks. We don't have
NOCOMMITs everywhere, just lots of places....

Original commit: elastic/x-pack-elasticsearch@d772cb1b96
2017-10-13 16:21:33 +00:00
Igor Motov e1d4fdc6d3 SQL: Unknown filter type [pattern_capture] warning in SQL IT Tests (elastic/x-pack-elasticsearch#2746)
Fixes java.lang.IllegalArgumentException: Unknown filter type [pattern_capture] for [email] warnings that pollute log files during execution of the AbstractSqlIntegTestCase-based tests.

Related to elastic/x-pack-elasticsearch#2501

Original commit: elastic/x-pack-elasticsearch@b9ec3c20d0
2017-10-13 11:55:48 -04:00
Lisa Cawley 84b00995e3 [DOCS] Removed ML GA limitation (elastic/x-pack-elasticsearch#2751)
Original commit: elastic/x-pack-elasticsearch@30aeb0d269
2017-10-13 16:52:40 +01:00
Nik Everett 99fea5f448 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@d521ecda35
2017-10-13 10:39:28 -04:00
Jason Tedor c35efb7adf Replace global checkpoint sync test
This commit replaces the REST test that the global checkpoint sync
action runs successfully as a privileged user. The test needs to be
replaced because it has a small race condition. Namely, the check that
the post-operation global checkpoint sync was successful could run
before the sync finishes running. To address this, we replace the REST
test with a test where we have a little more control and can assert busy
to avoid this race from failing the test.

Relates elastic/x-pack-elasticsearch#2749

Original commit: elastic/x-pack-elasticsearch@ea585b843c
2017-10-13 10:05:59 -04:00
Costin Leau 353c0c500b Fix and enable datetime tests (elastic/x-pack-elasticsearch#2680)
Firstly, data in H2 is now stored in TIMESTAMP WITH TIME ZONE since H2
does not allow a global TZ to be set and picks the JVM TZ when a record
is read.
JdbcAssert is now aware of this allows TIMESTAMP with TZ == TIMESTAMP

Discovered a serious bug in DateTimeFunction - unfortunately date
histogram is not useful except for year since most extract functions
avoid ordering which a histogram preserves.
Thus most DTF are now terms aggs with scripting.

Improved a bug that caused duplicate functions to not be detected because
of aliasing.

Moved some datetime tests to CSV but the aggs tests now are in sync with
H2

Fixed bug that caused arithmetic on aggs to not be properly resolved by
splitting the processor definition tree to aggName (unresolved) and
aggPath (resolved)

Original commit: elastic/x-pack-elasticsearch@e75ada68f1
2017-10-13 13:52:48 +03:00
Tanguy Leroux a6776cef97 [Monitoring] Add interval_ms to Monitoring documents (elastic/x-pack-elasticsearch#2650)
This commit adds a new interval_ms field to the monitoring documents. 
This field indicates the current collection interval for Elasticsearch or 
external monitored systems. The value is indexed as a long.

Related to elastic/x-pack-elasticsearch#212

Original commit: elastic/x-pack-elasticsearch@2ceb20455c
2017-10-13 11:18:47 +02:00
David Roberts 10cc0088e4 [DOCS] Make clearer that xpack.xyz.enabled settings are node settings (elastic/x-pack-elasticsearch#2731)
The discussion in elastic/x-pack-elasticsearch#2697 shows that this was not clear before.

relates elastic/x-pack-elasticsearch#2697

Original commit: elastic/x-pack-elasticsearch@87553faa2c
2017-10-13 09:22:21 +01:00
Costin Leau 2bbb86eff7 Eliminate NOCOMMITS per discussion
Original commit: elastic/x-pack-elasticsearch@bd091a6608
2017-10-13 00:33:16 +03:00
Costin Leau 21c8a9168b Add antlr to qa tests for IDE execution
Original commit: elastic/x-pack-elasticsearch@d146b98be3
2017-10-12 23:59:17 +03:00
Nik Everett 69d1a5c5dd Give audit logs more time to show up in SQL tests
CI has been failing since we had to remove our tweaks to the audit
logging configuration due to
https://github.com/elastic/x-pack-elasticsearch/issues/2705

So we increase the timeout. The timeout is very very long, but it
seems like we need it for CI which is often slow.

Original commit: elastic/x-pack-elasticsearch@91a926a031
2017-10-12 13:53:09 -04:00
Nik Everett 69a326dd94 Fix CliErrorsIT in sql security tests
It was disabled because the CLI didn't work with security but
we've since fixed that so we can enable it.

Original commit: elastic/x-pack-elasticsearch@8d9b5ad89b
2017-10-12 12:42:13 -04:00
Nik Everett 852af7de57 Fix error handling in SQL's CLI (elastic/x-pack-elasticsearch#2730)
We weren't returning errors correctly from the server
or catching them correctly in the CLI. This fixes that
and adds simple integration tests.

Original commit: elastic/x-pack-elasticsearch@259da0da6f
2017-10-12 16:32:15 +00:00
Jay Modi 9028c0a642 Allow PkiRealm to use truststore.password setting (elastic/x-pack-elasticsearch#2727)
This change fixes an incorrect check for a missing password setting for the PKI realm. The check
only allowed the secure setting to be used for the PkiRealm password even though the legacy setting
is still valid. This change fixes the check.

Relates elastic/x-pack-elasticsearch#2487

Original commit: elastic/x-pack-elasticsearch@a4524c2c05
2017-10-12 10:07:08 -06:00
Nik Everett 6478713304 Add support for username and password in SQL CLI (elastic/x-pack-elasticsearch#2718)
Add support for username and password in SQL CLI and adds tests that CLI works with security.

Original commit: elastic/x-pack-elasticsearch@39c8dbfc97
2017-10-12 15:55:29 +00:00
lcawley 7f37c2c431 [DOCS] Fixed broken link to java transport client
Original commit: elastic/x-pack-elasticsearch@f3036bb2a0
2017-10-12 14:29:20 +01:00
Jason Tedor aece28c286 Add test for global checkpoint sync with security
After a write operation on an index, a post-operation global checkpoint
sync fires. Previously, this action fired on the same user as executed
the write action. If the user did not have priviledges for this action,
the global checkpoint sync would fail. With an upstream change in core,
this action now fires as the system user. This commit adds a test that
create a user that has minimal write permissions on an index, but none
that would imply it could execute the global checkpoint sync. This then
serves as a test that the upstream change to fire the global checkpoint
sync as the system user is correct. This test must run as a mulit-node
test so that a replica is a assigned so that the global checkpoint sync
fires in the first place. This test does indeed fail without the
upstream change, and passes with it.

Relates elastic/x-pack-elasticsearch#2744

Original commit: elastic/x-pack-elasticsearch@bf7e771756
2017-10-12 09:19:17 -04:00
lcawley b628815dbe [DOCS] Fixed link to X-Pack transport client
Original commit: elastic/x-pack-elasticsearch@0870334e4b
2017-10-12 13:41:14 +01:00
Lisa Cawley 604229cd4d [DOCS] Added transport client info for X-Pack (elastic/x-pack-elasticsearch#2737)
* [DOCS] Added transport client info for X-Pack

* [DOCS] Relocated X-Pack java client info

* [DOCS] Added transport client deprecation info

Original commit: elastic/x-pack-elasticsearch@416aab1d76
2017-10-12 13:18:44 +01:00
Costin Leau fa4504ed28 Fix some NOCOMMITs
Original commit: elastic/x-pack-elasticsearch@1a6ac1e6c6
2017-10-12 14:24:56 +03:00
Costin Leau e3d072aeea Make JDBC driver throw only SQLException (elastic/x-pack-elasticsearch#2708)
A JDBC driver should throw only checked SQLExceptions.
Introduce JdbcSQLException and fix some no-commits along the way.

Original commit: elastic/x-pack-elasticsearch@299fcf9ace
2017-10-12 12:27:19 +03:00
Tanguy Leroux 0299886388 [Tests] Use XPack Usage API to verify Monitoring exporters are disabled (elastic/x-pack-elasticsearch#2648)
This commit changes the MonitoringIt and XPackRestIT tests so that the
disableMonitoring() method now use the XPack Usage API in order to check
that the monitoring exporters are correctly disabled. It checks at the
beginning of the tests (all exporters must be disabled before running
the test) and also at the end of the test.

This commit also fixes a bug in MonitoringIT where the Bulk thread pool
active queue was wrongly extracted from the response's map, forcing the
test to always wait for 30sec.

relates elastic/x-pack-elasticsearch#2459

Original commit: elastic/x-pack-elasticsearch@2d349e840f
2017-10-12 09:36:44 +02:00
Igor Motov 7f0c44f138 SQL: Fix string to boolean conversion (elastic/x-pack-elasticsearch#2728)
This commit also adds tests for conversion to boolean

Original commit: elastic/x-pack-elasticsearch@d7740929a2
2017-10-11 19:56:36 -04:00
Igor Motov 06b4c043b9 SQL: Add toXcontent test for SqlResponse (elastic/x-pack-elasticsearch#2726)
Original commit: elastic/x-pack-elasticsearch@aa7b8f1a91
2017-10-11 19:55:59 -04:00